Give a handler function rather than use defer?

[gregwebs]

Is it possible to have an API that does not rely on defer like the following?

func CopyFile(src, dst string) (err error) {
	defer err2.Handle(&err)

	wrapped := func(err) error {
		return fmt.Errorf("copy %s %s: %w", src, dst, err)
	}

	r := try.To1(os.Open(src), wrapped)
	defer r.Close()

	cleanup := func(err) error {
		_ = os.Remove(dst)
	}
	w := try.To1(os.Create(dst), wrapped, cleanup)
	defer w.Close()

	try.To1(io.Copy(w, r), wrapped)
	return nil
}

[gregwebs]

I added one defer required at the top of the function to catch the error that would certainly be necessary for a recovery.

[gregwebs]

Alternatively we could use the error value directly without needing to panic/recover?

func CopyFile(src, dst string) (err error) {
	wrapped := func(err) error {
		return fmt.Errorf("copy %s %s: %w", src, dst, err)
	}

	r := try.To1(&err, os.Open(src), wrapped)
	defer r.Close()

	cleanup := func(err) error {
		_ = os.Remove(dst)
	}
	w := try.To1(&err, os.Create(dst), wrapped, cleanup)
	defer w.Close()

	try.To1(&err, io.Copy(w, r), wrapped)
	return nil
}

[gregwebs]

oh, right, panic/recover is needed to make the return happen immediately.

[gregwebs]

Passing &err would allow for the possibility of returning a concrete type (that implements error) rather than only returning error.

[lainio]

Maybe I don't understand what you mean 🤔 but you can return concrete types or any error implementers, e.g., err2.Handle at the moment as well.

	defer err2.Handle(&err, func() {
		fmt.Println("cleaning target file")
		os.Remove(dst)
		err = new(TestError) // TestError is concrete type
	})

Is there something specific reason to avoid defer? Of course, there is some performance penalty but something else?

[gregwebs]

oh, I am still learning how Github discussions work :)

The point is to not have separate defer statements but instead connect the error handler directly with the code it is handling.

[gregwebs]

I have been able to implement the following on my fork:
Only one top-level defer is needed, and then handlers can be given directly to try.TryX functions.

        func(src, dst string) (err error) {
                defer err2.Handlef(&err, "copy %s %s", src, dst)

                r := try.Check1(os.Open(src))
                defer r.Close()

                rmFile := try.Cleanup(func() {
                        os.Remove(dst)
                })
                w := try.Try1(os.Create(dst))(rmFile, try.Fmt("os.Create %s", dst))
                defer w.Close()
                try.Try1(io.Copy(w, r))(rmFile)
                return nil
        }

[lainio]

Here's one example where we have inserted more information to error when the project goes forward. (The same file has others.)

Here's another example, maybe even more critical, where the short handler is called if any of the following try.To fails. And the handler is crucial because it sets the whole protocol state machine's state to failure.

Am I missing something, but is it so that with deferred handlers, you cannot forget, but if you aren't using defer and someone else adds new code, she must manually add the reference to clean up code to every new try.To call she makes?

One of my programming philosophies is to speed up the incremental-hacking cycle. That was one of the main reasons I started the err2 over three years ago. And that's why there exists, e.g., plain err2.Return. However, it's proven to be enough in many cases, and opposite to other Go projects, our projects don't have error stuttering: error: saving data: saving database: saving database row: SQL error in table: table error: SQL error in table row, you get the point.

Anyways, I have tried to find principles that support coding styles where you can start fast, and you still have error-save code, and by the lifetime of the code, you can make error messages even better, but only if you need to.

[lainio]

Now, I think I finally (sorry about that) understand why you don't "like" defer in err2 error handling scheme. Because the deferred lines only do something if the error value isn't nil. So, you think they are 'empty lines'. However, my point is that it's only how you want to read those lines. (And defer can be used just for checking, which is the case with recover())

    defer err2.Return(&err)

You could read that as it is: Defer error to return with this error pointer err until the whole function returns. By knowing that if err == nil, there is no error. It doesn't matter how the actual implementation is done or what functions are called inside. The end result is same, proper err value is returned in all cases.

    defer err2.Handle(&err, func() {...})

And you could read that: Defer error handling for error pointer err until the full function returns and then do this for it. Once again, by knowing that if err == nil, there is no error. It doesn't matter how the actual implementation is done or what functions are called, you can assume that the handler is not called if no error exists. Or you could even add an if statement at the beginning of the handler, and it would not break your code.

The guys and girls who have been using err2 long time love it. Also, those from other programming language cultures seem to understand the semantics right away. However, I don't claim you are wrong, but telling what we have learned.

[gregwebs]

First of all, I do really like what you have come up with. I would definitely like it better than the status quo. I think for those like myself there is a way to make an improved interface is all.

My issue is with programming by deferred pointer manipulation rather than just using plain old functions. The new API I am coming up with requires doing this only at the top-level of the function, and then after that one can use plain old functions and there is no need to reference pointers.

[gregwebs]

In some of the code I actually see a blank line between the defer and the try. But the two are dependent on each other: that doesn't make sense to me.

In existing Go code like that where we want to do some error handling and not just return the error, there is a flow of call function -> handle error. With err2, the flow is reversed to handle error -> call function. With the api I am developing, the flow is back to call function -> handle error.

[lainio]

I say we because I present our whole team that uses err2, and we think that deferred error handlers are some sort of error guardians. Pretty much as with Mutexes Unlocks:

func (m *Mgr) reopen(h *Handle) int {
	m.l.Lock()
	defer m.l.Unlock()

	if len(m.opened) < maxOpened {
		m.opened[h.cfg.UniqueID()] = h
		return h.reopen()
	}

	return m.closeOldestAndReopen(h)
}

I don't want to challenge you. I respect your opinion and what you have done, but I continue to explain what we have been learning.

In cases like this, it's excellent that to.Try isn't directly connected to error handling because now it's much easier to refactor.

	defer err2.Returnf(&err, "get credDef (%v) by root (%v)", cd.ID, ca.RootDid().Did())

	def := try.To1(vc.CredDefFromLedger(ca.RootDid().Did(), cd.ID))

It's very straightforward to bring e.g. new code to a function or method. Even copy&paste it from some other parts of the project or file:

	defer err2.Returnf(&err, "get credDef (%v) by root (%v)", cd.ID, ca.RootDid().Did())

	def := try.To1(vc.CredDefFromLedger(ca.RootDid().Did(), cd.ID))
	def := try.To1(vc.PreprocessCredDef(def)
	try.To(vc.CredDefIsValid(ca.RootDid().Did(), cd.ID, def))
	
	....

By using deferred error handlers we repeat ourselves less. And I'm a very refactor-oriented programmer if that's even a term 😄

[gregwebs]

Actually handlers aren't given directly to try.TryX functions because this is not possible. Instead try.TryX functions return a function that takes the handler aguments.

[lainio]

One reason not to use variadic functions is performance. Because Go's generics don't support them, you would need to use reflection or type assertion, which is extremely slow. It's n x 10x slower than without it. When err2 was started, there was a function err2.Try, which was variadic, and the last error had to be found by type assertion. It was about 30x slower than err2.Check, which was as fast as if err != nil. BTW, try.ToX is as fast as well.

The second reason is that you cannot freely mix variadic input and multiple return values.

[gregwebs]

Maybe I don't understand what you mean 🤔 but you can return concrete types or any error implementers, e.g., err2.Handle at the moment as well.

I am still learning generics, so I might be off-base here, but the point is basically that the existing API uses error everywhere and the values aren't connected by generics. At any rate I had to mostly give up on better specifying things with generics, it doesn't seem possible.

[gregwebs]

The second reason is that you cannot freely mix variadic input and multiple return values.

Yes, I think you can't mix anything with this actually. So my implementation of Try1 returns a function that error handlers can be applied to- it was the best I could do. It is a shame that Go doesn't formalize multiple return values as a tuple type.

[lainio]

Yes, the missing tuple is a pain. Also, support for variadic parametric types would help. I could drop Try1/2/3 only Try would be enough.

[gregwebs]

I would appreciate feedback on the API of my fork- the point is to not have separate defer statements but instead connect the error handler directly with the code it is handling.

[lainio]

Thanks for the results! But that's very weird. I have M1 Max, and I have the same results on macOS and Linux (Parallels virtualization, which, BTW, is usually far faster than native macOS on Go tests, go figure!). BTW, I got the same results with intel.

BenchmarkOldErrorCheckingWithIfClause-4   	1000000000	         0.3110 ns/op
BenchmarkOriginalTry-4                    	72113919	        15.48 ns/op
BenchmarkTry_ErrVar-4                     	1000000000	         0.3111 ns/op
BenchmarkTry_StringGenerics-4             	1000000000	         0.3109 ns/op <-- My version is as fast as..
BenchmarkTry_StrStrGenerics-4             	1000000000	         0.3116 ns/op <-- and this is too
BenchmarkCheckInsideCall-4                	1000000000	         0.3112 ns/op
BenchmarkCheckVarCall-4                   	1000000000	         0.3112 ns/op
BenchmarkCheck_ErrVar-4                   	1000000000	         0.3120 ns/op
BenchmarkCleanRecursionWithTryCall-4      	 1999972	       601.1 ns/op   <--- NOTICE this too!
BenchmarkRecursionWithCheckAndDefer-4     	  252932	      4552 ns/op
BenchmarkRecursionWithOldErrorCheck-4     	 1958605	       613.1 ns/op

NOTICE (recursion testing with non-local control flow) is interesting, especially for those who think exceptions are slow. (One of my backgrounds is C++ 3D graphics programming for game and CAD industries, where we learn this. Sadly Java programmers ruined exceptions or their reputation, at least).

[gregwebs]

My machine:

• Apple M1 Pro 32 GB Monterey 12.4
• go version: go1.19 darwin/arm64

My tests are suffixed with -8 instead of -4

[lainio]

That was my Linux virtual machine with 4 cores of 10 M1 max. When I run those benchmarks on my native Mac, the number is 10, but the results are otherwise the same with Linux.

I'll check your code later. Now I simply don't have time.

[lainio]

The reason/problem seems to be Go 1.19. It cannot inline (at least I think that's the reason) type parametrized functions with multiple return values. Should report this to somewhere 🤔

[lainio]

using export GOEXPERIMENT=unified restores the speed.

Luckily others have found it already

[gregwebs]

I am hopefully done with my fork now and I renamed the package to err3 now.
Highlights:

• API that minimizes the usage of defer and pointers to err error.
• combine deferred error handlers and try into one package but also make them available as separate packages
• don't offer stack printing, instead wrap errors with a stacktrace via pingcap/errors
• export stackprint as a separate package

Thanks for the great inspiration and code and test cases to make this possible. It was a lot easier to fork and make what I wanted then to convince you to change how things work. But I am willing to collaborate in ways that we agree make sense to lessen the forking or improve err2. It is possible to use the err3/try package with err2 right now. I also think that if you look at some of the API surface I have created for err3/err3 you might think of it as an improvement over the existing err2. For example, all of the handler functions are named Handle* and they consistently take &err as the first parameter.

[lainio]

Great work, and so fast!

I had time only to read the README for now, but I will try to be able to get back to this later. I must travel next week.

The idea to name all of the error handler helpers likeAnnotatex, Returnx, Handle, to just HandleX is something to consider 🤔

It's funny that all the people have their opinions. I asked some of the err2 users about parameter order and consistency over a year ago, and they wanted to keep it as it is.

Thank you for your time and effort. I will dive into this later!

[lainio]

Hello, I wanted to say that I appreciate all the work you have done here and with your Go error-handling proposals (two!). Awesome job 👍

I'm sorry that I have been so busy with other projects that I haven't been able to get back sooner. I have read your proposals and seen all your work with the fork.

It was especially bracing to get to know about the Zig programming language and how error handling is done in it. It was so nice to realize that what I currently have in the err2 package is precisely the same as how things are done with the Zig. For example, it has errdefer and all err2's deferred handlers precisely the same: they are called only if an error occurs.

Also, Zig has error tracing, which err2 has too, but the output is not exactly the same. One reason is to support Go's tools which are familiar with the current format. That can be seen with this playground, where those lines are highlighted where errors occur. I prefer automatic stack tracing instead of manual, which is easy to turn on even in production, but of course, I could still offer the manual API as well—something to consider, though.

I almost forget Zip has catch, which is if err... and no return in Go. And, of course, the try comes from err2/try. I use this opportunity and say that I hope you studied functions like try.isEOF, because they are handy.

Thanks to you! I have thought of the err2's API a lot. I have dropped the Annotate functions. I have also done some refactoring, even though I'm not yet sure if that much anti-DRY is necessary because you start to lose the readability at some point.

I haven't mentioned it earlier, but the current assertion package is a trendy part of the err2. Especially now because we can use it to replace testify or similar. The current err2 package is orthogonal on its own and with its sub-packages. And for that reason, I'm eager to hear any holistic suggestions about its API.

In the end, I'm not personally interested in taking part in the Go's proposals or conversations over there. I followed the try-proposal discussion and concluded that Go's proposal process is broken and even misleading. What a waste of good people's work and time! Especially when we are talking about the error-handling, they should understand that "one size doesn't fit all." Automatic memory management and manual error propagation don't belong together.

[gregwebs]

Thanks for the encouragement.

I have done some major refactoring on my fork, but underneath it all the core error recovery is still the same. Except I added the ability to recover a panic and add an annotation to it. In my fork, nothing is printed, instead errors and panics are annotated.

I also changed the API to not return any values. So

x, err := f()
try.Check(err)

It is slightly more verbose, but I think it works well, and actually works better when f() is large and when you want to annotate an error. Also, I think it is more acceptable to open-minded Go programmers. Another nice aspect is that the library no longer needs generics (I am just using them for one unnecessary helper function) and otherwise the implementation is more straightforward. Just the one recovery function is tricky now, thanks for the test cases, I wouldn't have been able to make my changes without them.

I just committed a refactoring so that the error throwing functions are just Check, Checkf, and Checkw.

I hold out some hope for this proposal and am considering instead of using the Check functions of this library adopting this pattern I noticed in the standard library. Regardless, the Handle functions are quite useful to be able to annotate both errors and panics.

[lainio]

Give a handler function

Hi,

I thought that I let you know that your "feature request" is now possible. However, one:

    defer err2.Handle(&err)

is needed per function where error handling is happening. But now err2/try support try.Out functions that implement discriminated union type and sort of Error Handling Language (i.e. domain-specific language). Here's the link to the documentation and complete sample of CopyFile.

Now, every Zig error handling feature is supported by the err2 package.