Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Ferature Request] Allow Stream transforms to be chained at the chunk level #15828

Closed
DartBot opened this issue Dec 30, 2013 · 1 comment
Closed
Labels
area-library closed-obsolete Closed as the reported issue is no longer relevant type-enhancement A request for a change that isn't a bug

Comments

@DartBot
Copy link

DartBot commented Dec 30, 2013

This issue was originally filed by gruyere.emmen...@gmail.com


What steps will reproduce the problem?

This is based on a question I put in on SO
http://stackoverflow.com/q/20815913/1085699

Currently:
Stream<List<int>> stream = new File('Data.txt').openRead();

   stream
      .transform(const Utf8Decoder())
        .transform(const LineSplitter())
          .listen((line) {
            stdout.writeln(line);
          }).asFuture().catchError(() => print());

Will call each transform after reading the entire contents of the file first. This gives the user very little control over the amount of work being performed and is not conducive to chunking into efficient pieces that can handled quickly.

What would be better?
The current "Chain of Responsibility" pattern does not give enough control to the user. It would be better to use a function parameter for the next downstream transform so that it can be called repeatedly when the data chunk size is reached. Some transforms seem to have the size of the chunk implicit in their definition (LineSplitter). I think this library needs to be rethought. I would point to Node libraries for streaming: https://github.com/jahewson/node-byline

Dart Editor version 1.0.0_r30798 (STABLE)
Dart SDK version 1.0.0.10_r30798

@lrhn
Copy link
Member

lrhn commented Jan 2, 2014

If each converter allows chunked conversion, the transform Stream should recognize this and fuse the converters.

We do want to do this automatically where possible.

Until then, you can fuse the converters manually:
 stream.transform(UTF8.decoder.fuse(const LineSplitter())
       .listen(stdout.writeln)
       .asFuture().catchError(print);


cc @floitschG.
Removed Type-Defect label.
Added Type-Enhancement, Area-Library, Triaged labels.

@kevmoo kevmoo added type-enhancement A request for a change that isn't a bug and removed priority-unassigned labels Feb 29, 2016
@lrhn lrhn added the core-m label Aug 11, 2017
@floitschG floitschG added closed-obsolete Closed as the reported issue is no longer relevant and removed core-m labels Aug 28, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area-library closed-obsolete Closed as the reported issue is no longer relevant type-enhancement A request for a change that isn't a bug
Projects
None yet
Development

No branches or pull requests

4 participants