Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Early close resource file descriptors #95

Merged
merged 6 commits into from
Aug 20, 2024
Merged

Early close resource file descriptors #95

merged 6 commits into from
Aug 20, 2024

Conversation

ymmt2005
Copy link
Member

@ymmt2005 ymmt2005 commented Aug 13, 2024

Previously, the file descriptor in a cybozu::resource remained open
throughout its lifetime, so it was unnecessary to protect references
to such a file descriptor. However, this sometimes caused out of
file descriptors error due to its late closing.

This PR introduces a new API design around cybozu::resource so that
the reactor can close file descriptors earlier than before.

The new API design consists of the following notable changes:

  • cybozu::resource::m_fd is now a private member, so no direct reference is allowed.
  • cybozu::resource holds a readers-writer lock to protect m_fd.
  • Subclasses should use resource::with_fd to get the file descriptor along with a reader lock.
  • The reactor thread tries to close the file descriptor early only if it can get a writer lock.

Resolve: #93

@ymmt2005 ymmt2005 self-assigned this Aug 13, 2024
cybozu/reactor.hpp Outdated Show resolved Hide resolved
cybozu/reactor.hpp Outdated Show resolved Hide resolved
cybozu/reactor.hpp Show resolved Hide resolved
cybozu/reactor.hpp Show resolved Hide resolved
cybozu/reactor.hpp Outdated Show resolved Hide resolved
Previously, the file descriptor in a `cybozu::resource` remains open
throughout its lifetime, so it was unnecessary to protect the reference
to such a file descriptor.

The new design is to allow the reactor to close resource file
descriptors early before the resource is destructed so that yrmcdsd
will be less likely to cause an out of file descriptors error.

For this, we have to proctect a resource file descriptor not to
be closed while it is being used. The new API design implements this
with the following changes:

- `resource::m_fd` is now a private member, so no direct reference is allowed.
- Subclasses should use `resource::with_fd` to get the file descriptor along with a reader lock.
- The reactor thread closes the file descriptor only if it can get a writer lock.
@ymmt2005 ymmt2005 marked this pull request as ready for review August 19, 2024 07:03
cybozu/reactor.hpp Outdated Show resolved Hide resolved
cybozu/reactor.hpp Outdated Show resolved Hide resolved
cybozu/reactor.hpp Show resolved Hide resolved
cybozu/reactor.hpp Outdated Show resolved Hide resolved
src/memcache/memcache.cpp Show resolved Hide resolved
cybozu/tcp.hpp Outdated Show resolved Hide resolved
cybozu/tcp.hpp Show resolved Hide resolved
cybozu/tcp.hpp Show resolved Hide resolved
cybozu/reactor.hpp Outdated Show resolved Hide resolved
cybozu/reactor.hpp Show resolved Hide resolved
src/memcache/sockets.cpp Outdated Show resolved Hide resolved
@ymmt2005
Copy link
Member Author

@nojima
Addressed all review comments. Please take another look.

@ymmt2005 ymmt2005 requested a review from nojima August 19, 2024 14:36
if( ! valid() ) return false;
// no need to check m_closed because it becomes true only after m_valid is set to false.

if( std::forward<Func>(f)(m_fd) ) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't need the perfect forwarding for f because f is not forwarded to any function, IMO. Is it sufficient to write it as f(m_fd)?

Copy link
Member Author

@ymmt2005 ymmt2005 Aug 20, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I referenced this https://www.reddit.com/r/cpp/comments/kzvjgn/comment/gjracal/
The generated assembly looks better if we use the perfect forwarding.
But TBH, I'm not very sure...

If it does not do any harm, I'd like to keep the current way.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I could not find a better assembly. Moreover, when using std::forward, an extra call is added.
(I checked this https://godbolt.org/z/4svKjq )

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@toshipp oops, I misread. will fix.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

It generated worse code.
toshipp
toshipp previously approved these changes Aug 20, 2024
Copy link
Collaborator

@toshipp toshipp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

Copy link
Collaborator

@nojima nojima left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good to me

@ymmt2005
Copy link
Member Author

Thank you for your reviews.

@ymmt2005 ymmt2005 merged commit 8f4c193 into master Aug 20, 2024
6 checks passed
@ymmt2005 ymmt2005 deleted the early-close-2 branch August 20, 2024 06:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Close socket file descriptors earlier
3 participants