feat: add deduplicate parameter to create topic using cli

[fraidev]

Solves #4388

[ajhunyady]

@fraidev, we should rename deduplicate to dedup, for easier typing.

I also think we should expose 2 variants:

1. PREFERRED: Add a convenience command that will download the fluvio/[email protected] smartmodule automatically.

   fluvio topic create dedup_topic --dedup
   fluvio topic create dedup_topic --dedup --dedup-age 1h --dedup-count 5 

2. ADVANCED: Expose the same --smartmodule command that works with fluvio consume for the user to apply a user-defined smartmodules and parameters:

   fluvio hub sm download fluvio/[email protected]

    fluvio topic create dedup_topic --smartmodule fluvio/[email protected]
    fluvio topic create dedup_topic --smartmodule fluvio/[email protected] -e dedup-page=1h  -e dedup-count=5

[fraidev]

2. --smartmodule

@fraidev, we should rename deduplicate to dedup, for easier typing.

I also think we should expose 2 variants:

1. PREFERRED: Add a convenience command that will download the fluvio/[email protected] smartmodule automatically.

   fluvio topic create dedup_topic --dedup
   fluvio topic create dedup_topic --dedup --dedup-age 1h --dedup-count 5 

2. ADVANCED: Expose the same --smartmodule command that works with fluvio consume for the user to apply a user-defined smartmodules and parameters:

   fluvio hub sm download fluvio/[email protected]

    fluvio topic create dedup_topic --smartmodule fluvio/[email protected]
    fluvio topic create dedup_topic --smartmodule fluvio/[email protected] -e dedup-page=1h  -e dedup-count=5

Sure, I did it like the PREFERRED way, and I also created an issue for the ADVANCED way

[sehz]

Is there issue describing what new option does?

[sehz]

Is there contract on smartmodule name and version? What if user doesn't download right version of smartmodule

[sehz]

this should be builder pattern so can customize this

[fraidev]

I noticed that Deduplication, Bounds and Filter implements Builder, but seems a little verbose to create 3 Builders for this, no?

Maybe with additional methods at Deduplication?

[ajhunyady]

Is there contract on smartmodule name and version? What if user doesn't download right version of smartmodule

As per my recommendation, there are 2 flavors (see above).
for the

1. preferred - we should decide the version
2. advanced - it' just like any other smartmodule, it's up to the user.

[sehz]

--dedup --dedup-age 1h --dedup-count 5 

Is this option applies to all deduplication smartmodule or just on specific version of smartmodule?

[sehz]

@fraidev
Let's have issue for it to iterate instead of here.

[fraidev]

--dedup --dedup-age 1h --dedup-count 5 

Is this option applies to all deduplication smartmodule or just on specific version of smartmodule?

Specific version.
"fluvio/[email protected]" for now.

[fraidev]

@fraidev Let's have issue for it to iterate instead of here.

Sure, it's here: #4388

[fraidev]

@sehz I changed the PR to download the specific SM if not downloaded.