Feature: programmatic startup/shutdown improvements

[XVincentX]

While writing integration tests for plugin I wish I had 3 main features in the gateway:

1. A way to shutdown a gateway instance (mainly for testing purposes, when you want to cleanup the resources).

gatewayInstance.close().then(()=>console.log("It's over"));

2. A way to know that the gateway is ready to get requests; the run method might return a Promise

gatewayInstance.run().then(()=>console.log("Time to proxy requests"));

3. A way to feed the gateway configuration from the code directly, and not being constrained to give file paths or directories.

const systemConfig = {};
const gatewayConfig = {}

gateway().load(systemConfig, gatewayConfig).run().then(...)

[DrMegavolt]

@XVincentX check out gateway.helper in tests
startGatewayInstance is here

and admin.helper for admin server

[DrMegavolt]

and config from code is not a problem gateway({config})

[kevinswiber]

One issue here is that configuration is a global singleton. If you have multiple gateway().load() calls, the last config will win.

[XVincentX]

@DrMegavolt

I’ve been looking to the helpers you provided and I realized that they’re basically miming somebody starting the gateway from the command line - and then throw some requests and wait the gateway to respond correctly in order to make sure it’s ready.

My aim is more - let’s provide a builtin mechanism to start and stop the gateway, and I’ve started look into this. It turns out we can do that kind of easily.

I've been playing around with a branch here:
https://github.com/ExpressGateway/express-gateway/compare/feature/programmatic-startup-shutdown?expand=1

The idea is - instead of returning this from the run method - we could collect and return the inner promises and use these as a way to say “the gateway is ready” simply using the then method.

Same goes with an eventual shutdown method - just close all the applications and resolve the promises once.

Although we have that singleton problem right now, it can be tamed with a correct usage of beforeEach.

//cc @kevinswiber

What do you guys think?

[DrMegavolt]

@kevinswiber this is why config is no just a parameter for gateway and for admin
https://github.com/ExpressGateway/express-gateway/blob/master/lib/gateway/index.js#L63

@XVincentX here is this part implemented for admin
https://github.com/ExpressGateway/express-gateway/blob/master/test/common/admin-helper.js
it is exactly as you described

please do not join it with gateway. it is common you need only one of those

for gateway
check this part
https://github.com/ExpressGateway/express-gateway/blob/master/test/common/gateway.helper.js#L24

and yes it can subscribe to http-ready instead of doing 25times request and waiting for 404
and it was built for full e2e scenario

There is alternative variant
https://github.com/ExpressGateway/express-gateway/blob/master/test/common/routing.helper.js

it is actually closer to admin-helper. it can start\stop gateway.
obviously it is mixed with wrapper around superset for common scenarios and can be extracted

[DrMegavolt]

Also there are some complex but common scenarios

Start fake backend (with request handler controlled by test)
Start gateway with config to proxy to this server

register policy (controlled by test)
start eg with this policy

start eg,
start admin
start fake backend

[DrMegavolt]

and one more thing:
you will find a lot of tests with code

let config  = require(lib/config)
originalConfig = config.gatewayConfig

//tests

config.gatewayConfig = originalConfig
gateway() ... 

this is no longer needed. you can pass the config directly to gateway.

let Config  = require(lib/config/config)
let config = new Config()
config.gatewayConfig = {}

gateway({config})

[XVincentX]

Note to self — with the introduction of validation schemas we should eventually remember to clear those when the gateway is shutting down. If not, we might get annoyances in tests such as a0868cf

[toryalsip]

I'm curious if there are further plans for this in the near future or if it's going to be on the back-burner for a while? I asked this because I'm thinking about ways as a plugin developer to easily test plugins. With a bit of digging around in the source code, and seeing what patterns exist currently within the express-gateway/tests folder, I was able to figure out how to do what I'm thinking but it currently takes a fair amount of boilerplate code to do it. Could a set of test tools that essentially reduces that boilerplate be of value?

[XVincentX]

Hey @jbalsip,
it's good to have such activity on this issue — it means that EG is getting traction and people are willing to write external plugins.

So — we had several pull requests that improved the situation. There's still some work to do but we definitely want to get there.

Right now I've been testing the plugins by putting express-gateway as a devDependency and then installing the plugin source code into it. Once the gateway has started, I throw some http requests and I check for the output.

You can see the relevant parts in the rewrite plugin: Plugin installation, starting the gateway from the CI server, sending Http request to the gateway

It is not optimal, but so far has the best quality/time ratio.

I have a plan for that, but before we have to clean up some technical debt we have in place (such as #610, for example); and then starting towards this road.

Another item that is hurting EG's testability, for example, is the fact that it can't be initialised providing the configuration as a JSON Object (at least using the public API we offer). You're forced to provide a configuration path that means — files on the disk.

I'm glad you brought you up this though.

//cc @kevinswiber @altsang Maybe we can prioritise this item?

What we can do right now — it's to document the process so that users do not feel lost like you are right now.

[DrMegavolt]

@jbalsip nice work
https://github.com/jbalsip/express-gateway-test-tools

If we go as @XVincentX suggesting by putting express-gateway as a devDependency
Will it make sense to just do some cleanup on existing helpers (which are not perfect)?
and some stub generators like you have for config in your repo

so in actual plugin test it will be just require('express-gateway/test/helpers')

btw it doesn't have to be migration of all existing tests at once to new helpers.

[toryalsip]

Thank you @DrMegavolt . I think having helpers and stub generators can go a long way in those efforts, and pretty much what I'd been envisioning with what I've been doing in the test tools repo.

I did see what was done with the rewrite plugin but found when I tried out something that looked almost identical that it didn't quite work for me. I think circleci was trying to execute my tests while the gateway was still starting up. I might've missed some nuance in configuring but didn't have much chance to delve into that. So instead I started digging more into how the tests could control that more closely, and I've had fairly good results. Only drawback I've seen so far, aside from reducing the code it takes to do it, has been since the gateway is running in the same process as the tests that the console output will be mixed between gateway logging and test results. That's probably a whole other matter and perhaps out-of-scope for this particular issue, and I'm sure there's probably some way I can work around that somehow as well.

At any rate, I'm certainly very interested in this and I'd be glad to help out as time allows. I'm actually trying to put together a rough proof of concept to demo to show my team sometime, but hard to say when that will be ready.

[XVincentX]

Please keep us posted on that. We'll try to have an internal discussion as well and see if we can invest some time in test helpers.

Thanks again!

[DrMegavolt]

@jbalsip
quick comment on logs

express-gateway/lib/plugins.js

Line 65 in ca7b7af

this.logger = logger;

in plugin context a logger is passed. a special one for plugins.
It log level is controlled by process.env.LOG_LEVEL_PLUGINS
https://github.com/ExpressGateway/express-gateway/blob/master/lib/logger.js#L48

so logs will not be mixed LOG_LEVEL_PLUGINS=debug (internal policies use LOG_LEVEL_POLICY)
so you should see mostly logs from your plugin and other custom plugins

or use some lib like debug to completely separate output

---

+1 to see the tests from your PoC

Our current helpers cover scenarios of running EG in the same process and as a separate process (looks ugly but seems to work)
We use separate process to run E2E tests, but those helpers were never utilized in external plugin so it is hard to tell if it actually works correctly

Feel free to make small PRs with helpers as you see it. At some point the picture of helpers will be more or less complete so we document the approach as beta feature on the EG.io site and in the release notes. I believe feedback from others will follow, so the helpers can be improved and moved out of beta status.

beta is mostly to avoid migration issues. One can always reference specific version of EG in their plugin and do migration in there own pace.

Thanks!

[toryalsip]

That is very helpful @DrMegavolt and thank you. I'll be sure to incorporate that into what I'm working on.