Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bed track opening is too slow when file is on a SSHFS disk #27

Open
iromeo opened this issue Oct 24, 2018 · 3 comments
Open

Bed track opening is too slow when file is on a SSHFS disk #27

iromeo opened this issue Oct 24, 2018 · 3 comments

Comments

@iromeo
Copy link
Contributor

iromeo commented Oct 24, 2018

Bed track opening is too slow when file is on a SSHFS disk

E.g. try to open /mnt/stripe/bio/experiments/blueprint/data_redo/bp.all.covered.regions.hg19.filtered.mean10.bed.

IO reading speed is about 50-100 kb/s or less.

Probably it is caused by my current network connection conditions. Let's check it later.
Is related to #1302.

@iromeo
Copy link
Contributor Author

iromeo commented Oct 24, 2018

Consider trying

fun Path.inputStream(vararg options: OpenOption) =
        FileInputStream(toFile()).buffered(32 * 1024).streamFor(name)

instead of

fun Path.inputStream(vararg options: OpenOption) =
        Files.newInputStream(this, *options).buffered().streamFor(name)

There are some reports that seek operation is slow while using Files.newInputStream(this, *options), althoug shouldn't affect us due to reading file from the begining up to end in Bed parser.

@iromeo
Copy link
Contributor Author

iromeo commented Oct 24, 2018

Also, consider custom buffer size in BufferedReader / BufferedInputStream, the default value is 8 kb

@iromeo
Copy link
Contributor Author

iromeo commented Oct 24, 2018

related: 2ad3a4619de0c5dd21ef83e342b2e95e05d75629

@olegs olegs transferred this issue from another repository Mar 21, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant