Allow to pass limit as null to not break the pipeline

[norberttech]

Change Log

Added

Fixed

Changed

• Allow to pass limit as null to avoid breaking a pipeline

Removed

Deprecated

Security

---

Description

Now when limit needs to come from a user input, we need to split the pipeline in order to check if limit is not null.

$df = df()
  ->read(from_());

if ($limit !== null) { 
  $df->limit($limit);
}

$df->run()

After we merge this PR it wont be necessary anymore:

if ($limit < 0) {
   $limit = null;
}

$df = df()
  ->read(from_())
  ->limit($limit)
  ->run();

[github-actions]

Flow PHP - Benchmarks

^{_{Results of the benchmarks from this PR are compared with the results from 1.x branch.}}

Extractors

+-----------------------+-------------------+------+-----+-----------------+------------------+-----------------+
| benchmark             | subject           | revs | its | mem_peak        | mode             | rstdev          |
+-----------------------+-------------------+------+-----+-----------------+------------------+-----------------+
| CSVExtractorBench     | bench_extract_10k | 1    | 3   | 4.848mb +0.01%  | 552.619ms +0.52% | ±0.40% -70.44%  |
| JsonExtractorBench    | bench_extract_10k | 1    | 3   | 4.921mb +0.01%  | 1.063s -0.38%    | ±1.61% +107.32% |
| ParquetExtractorBench | bench_extract_10k | 1    | 3   | 86.372mb +0.00% | 905.902ms +2.16% | ±0.54% +232.09% |
| TextExtractorBench    | bench_extract_10k | 1    | 3   | 4.578mb +0.01%  | 36.598ms +3.09%  | ±0.88% -16.60%  |
| XmlExtractorBench     | bench_extract_10k | 1    | 3   | 4.553mb +0.01%  | 602.436ms -1.16% | ±0.26% +382.78% |
+-----------------------+-------------------+------+-----+-----------------+------------------+-----------------+

Transformers

+-----------------------------+--------------------------+------+-----+------------------+-----------------+----------------+
| benchmark                   | subject                  | revs | its | mem_peak         | mode            | rstdev         |
+-----------------------------+--------------------------+------+-----+------------------+-----------------+----------------+
| RenameEntryTransformerBench | bench_transform_10k_rows | 1    | 3   | 127.375mb +0.00% | 72.263ms +2.96% | ±0.62% -58.95% |
+-----------------------------+--------------------------+------+-----+------------------+-----------------+----------------+

Loaders

+--------------------+----------------+------+-----+------------------+------------------+----------------+
| benchmark          | subject        | revs | its | mem_peak         | mode             | rstdev         |
+--------------------+----------------+------+-----+------------------+------------------+----------------+
| CSVLoaderBench     | bench_load_10k | 1    | 3   | 64.047mb +0.00%  | 107.116ms +4.14% | ±0.63% +62.11% |
| JsonLoaderBench    | bench_load_10k | 1    | 3   | 84.394mb +0.00%  | 100.596ms +5.65% | ±0.70% -57.53% |
| ParquetLoaderBench | bench_load_10k | 1    | 3   | 161.243mb +0.00% | 20.740s +1.18%   | ±0.34% -2.98%  |
| TextLoaderBench    | bench_load_10k | 1    | 3   | 18.110mb +0.00%  | 31.397ms +6.93%  | ±0.35% -71.65% |
+--------------------+----------------+------+-----+------------------+------------------+----------------+

Building Blocks

+-------------------+----------------------------+------+-----+------------------+------------------+-----------------+
| benchmark         | subject                    | revs | its | mem_peak         | mode             | rstdev          |
+-------------------+----------------------------+------+-----+------------------+------------------+-----------------+
| EntryFactoryBench | bench_entry_factory        | 1    | 3   | 105.994mb +0.00% | 458.568ms +0.93% | ±1.21% +27.03%  |
| EntryFactoryBench | bench_entry_factory        | 1    | 3   | 55.184mb +0.00%  | 230.606ms +0.74% | ±1.78% +358.36% |
| EntryFactoryBench | bench_entry_factory        | 1    | 3   | 14.706mb +0.00%  | 50.586ms +1.08%  | ±0.16% -81.90%  |
| TypeDetectorBench | bench_type_detector        | 1    | 3   | 43.894mb +0.00%  | 370.293ms +3.68% | ±1.34% +64.48%  |
| TypeDetectorBench | bench_type_detector        | 1    | 3   | 11.704mb +0.00%  | 71.962ms +0.11%  | ±1.19% +19.13%  |
| RowsBench         | bench_chunk_10_on_10k      | 2    | 3   | 97.037mb +0.00%  | 3.335ms +5.09%   | ±0.87% -52.66%  |
| RowsBench         | bench_diff_left_1k_on_10k  | 2    | 3   | 114.324mb +0.00% | 183.487ms +0.87% | ±0.58% +13.33%  |
| RowsBench         | bench_diff_right_1k_on_10k | 2    | 3   | 97.044mb +0.00%  | 18.735ms +2.58%  | ±0.44% +91.13%  |
| RowsBench         | bench_drop_1k_on_10k       | 2    | 3   | 97.912mb +0.00%  | 1.665ms +16.64%  | ±2.45% -4.61%   |
| RowsBench         | bench_drop_right_1k_on_10k | 2    | 3   | 97.912mb +0.00%  | 1.613ms +12.52%  | ±2.92% +182.36% |
| RowsBench         | bench_entries_on_10k       | 2    | 3   | 96.073mb +0.00%  | 4.774ms +4.31%   | ±2.41% +298.52% |
| RowsBench         | bench_filter_on_10k        | 2    | 3   | 96.602mb +0.00%  | 17.226ms +2.66%  | ±0.90% -44.28%  |
| RowsBench         | bench_find_on_10k          | 2    | 3   | 96.602mb +0.00%  | 17.042ms +2.90%  | ±1.50% +511.53% |
| RowsBench         | bench_find_one_on_10k      | 10   | 3   | 95.293mb +0.00%  | 1.994μs +5.28%   | ±2.40% -5.08%   |
| RowsBench         | bench_first_on_10k         | 10   | 3   | 95.293mb +0.00%  | 0.400μs 0.00%    | ±0.00% 0.00%    |
| RowsBench         | bench_flat_map_on_1k       | 2    | 3   | 104.511mb +0.00% | 14.609ms +1.79%  | ±0.96% +87.88%  |
| RowsBench         | bench_map_on_10k           | 2    | 3   | 134.578mb +0.00% | 73.363ms +1.12%  | ±0.51% -71.44%  |
| RowsBench         | bench_merge_1k_on_10k      | 2    | 3   | 97.121mb +0.00%  | 1.376ms +1.43%   | ±2.79% -12.89%  |
| RowsBench         | bench_partition_by_on_10k  | 2    | 3   | 100.424mb +0.00% | 66.888ms +2.73%  | ±1.19% -43.00%  |
| RowsBench         | bench_remove_on_10k        | 2    | 3   | 98.174mb +0.00%  | 3.860ms +2.17%   | ±1.62% -33.27%  |
| RowsBench         | bench_sort_asc_on_1k       | 2    | 3   | 95.586mb +0.00%  | 42.174ms +0.84%  | ±0.90% +75.19%  |
| RowsBench         | bench_sort_by_on_1k        | 2    | 3   | 95.587mb +0.00%  | 43.023ms +1.83%  | ±2.81% +250.54% |
| RowsBench         | bench_sort_desc_on_1k      | 2    | 3   | 95.586mb +0.00%  | 45.377ms +7.53%  | ±3.25% +231.13% |
| RowsBench         | bench_sort_entries_on_1k   | 2    | 3   | 97.733mb +0.00%  | 8.707ms +6.74%   | ±3.15% +537.61% |
| RowsBench         | bench_sort_on_1k           | 2    | 3   | 95.483mb +0.00%  | 29.018ms -0.42%  | ±0.47% -3.46%   |
| RowsBench         | bench_take_1k_on_10k       | 10   | 3   | 95.293mb +0.00%  | 13.920μs -0.57%  | ±1.23% +5.34%   |
| RowsBench         | bench_take_right_1k_on_10k | 10   | 3   | 95.293mb +0.00%  | 15.200μs -0.57%  | ±0.00% -100.00% |
| RowsBench         | bench_unique_on_1k         | 2    | 3   | 114.325mb +0.00% | 191.415ms +0.80% | ±1.02% +181.64% |
+-------------------+----------------------------+------+-----+------------------+------------------+-----------------+

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 83.01%. Comparing base (c851044) to head (265f3cf).
Report is 2 commits behind head on 1.x.

Additional details and impacted files

@@           Coverage Diff           @@
##              1.x    #1498   +/-   ##
=======================================
  Coverage   83.01%   83.01%           
=======================================
  Files         683      683           
  Lines       18280    18282    +2     
=======================================
+ Hits        15175    15177    +2     
  Misses       3105     3105           

Components	Coverage Δ
etl	86.04% <100.00%> (+<0.01%)	⬆️
cli	86.73% <ø> (ø)
lib-array-dot	94.53% <ø> (ø)
lib-azure-sdk	62.56% <ø> (ø)
lib-doctrine-dbal-bulk	90.11% <ø> (ø)
lib-filesystem	78.02% <ø> (ø)
lib-parquet	84.33% <ø> (ø)
lib-parquet-viewer	82.02% <ø> (ø)
lib-rdsl	87.09% <ø> (ø)
lib-snappy	90.69% <ø> (ø)
bridge-filesystem-async-aws	90.38% <ø> (ø)
bridge-filesystem-azure	89.92% <ø> (ø)
bridge-monolog-http	96.38% <ø> (ø)
symfony-http-foundation	72.72% <ø> (ø)
adapter-chartjs	86.45% <ø> (ø)
adapter-csv	89.57% <ø> (ø)
adapter-doctrine	88.08% <ø> (ø)
adapter-elasticsearch	97.19% <ø> (ø)
adapter-google-sheet	78.04% <ø> (ø)
adapter-http	59.15% <ø> (ø)
adapter-json	90.62% <ø> (ø)
adapter-logger	53.84% <ø> (ø)
adapter-meilisearch	97.75% <ø> (ø)
adapter-parquet	80.85% <ø> (ø)
adapter-text	84.44% <ø> (ø)
adapter-xml	83.15% <ø> (ø)