Go-dog code coverage percentage shows 0.0%

[Kousik1612]

Please answer these questions before submitting a bug report.

What version of godog are you using?

0.8.0

What version of Go are you using?

1.15.7

What did you do?

I'm trying to fetch code coverage on godog feature files(functional test cases). Have executed the below command and made necessary changes to testmain package for integration of godog with go test.
cd functionaltests
go test -v --godog.format=pretty -coverprofile=functionaltests.cov -covermode=atomic >> fcoverageresult.txt
All my test cases ran and i get the below output:

801 scenarios ([32m782 passed[0m, [31m19 failed[0m) 7066 steps ([32m7008 passed[0m, [31m19 failed[0m, [36m39 skipped[0m) 1m1.610143019s

What did you expect to see?

90% code coverage

What did you see instead?

coverage: 0.0% of statements

Additional context

Tried using httptest instead of http as well in the test packages. Still facing the same issue. Any suggestions?

[feyyazesat]

Same for me as well,
I tried to separate unit-tests and e2e test runs from each other, the entry point is gotest.

Godog version

v0.11.0

Go version

go1.15.6 darwin/amd64

// file: main_setup_test.go
// this shows %0 test coverage
func TestMain(m *testing.M) {

	// Check whether should it run godog E2E tests or not, this is the entry point for e2e tests
        // I always pass arguments like godog.random, I'm aware it is super hacky :/
	for _, argStr := range os.Args[2:] {
		if isE2E := strings.Contains(argStr, "godog"); isE2E == true {
			pflag.Parse()
			os.Exit(runE2E()) // the changed line
		}
	}

	// Otherwise run native test suite
	os.Exit(m.Run())
}

// file: main_unit_test.go
func TestItDoesSomething(t *testing.T) {
    ...
}

...
...

// file: main_e2e_test.go
func runE2E() int {
	opts.Paths = []string{"features"}

	return godog.TestSuite{
		Name:                 "E2E Test Suite",
		TestSuiteInitializer: InitializeTestSuite,
		ScenarioInitializer:  InitializeScenario,
		Options:              &opts,
	}.Run()
}

...
...

if I apply following changes to main_setup_test.go:

// file: main_setup_test.go
// Then I suddenly get %86 coverage

// os.Exit(runE2E()) changing this to
status := runE2E()
if s := m.Run(); s > status {
    status = s
}
os.Exit(status)
}

Nevertheless, I still cannot get the coverage report. If I only run unit-tests/native tests then I can receive coverage reports as well.

I execute my tests like below:

// for E2E tests
 go test  -v -race -coverprofile=../../tmp/e2e.cov -covermode=atomic --godog.random
// for Unit tests
 go test  -v -race -coverprofile=../../tmp/unit.cov -covermode=atomic

// e2e.cov file only contains following line either I add m.Run or not doesn't change the result
mode: atomic
-----------------

// unit.cov contains the following line + many lines regarding the coverage
mode: atomic
...
...
...

[vearutop]

I think the issue you're seeing is caused by default behavior of Go, which collects coverage only in the package that owns the test, this makes sense for unit tests. For end-to-end (integration) tests this is usually not the desired behavior.

Please try configuring which packages should qualify for coverage with -coverpkg ./... and check if helps getting coverage.

[nhatthm]

My case is a bit different, my project has this structure

├─ features/
│  ├─ bootstrap/
│  |  └─ godog_test.go
|  └─ xxx.feature
├─ file1.go
└─ file2.go

Test suite is in features/bootstrap/godog_test.go

func init() {
	godog.BindCommandLineFlags("godog.", &opt)
}

func TestMain(m *testing.M) {
	pflag.Parse()

	os.Exit(m.Run())
}

func TestIntegration(t *testing.T) {
	// Test suite is in here.
}

When I run

go test ./features/... -gcflags=-l -coverprofile=features.coverprofile -coverpkg ./... -race -v

I have 0% coverage in coverprofile file

However, when I put flag.Parse (the Go flag) in the TestMain, like this:

func TestMain(m *testing.M) {
	flag.Parse()
	pflag.Parse()

	os.Exit(m.Run())
}

It works! And I have the right report in the coverprofile file.

I have no idea what happens 🤷‍♂️

[ds0nt]

Just wondering if your tests are panicking. Go test coverage will not generate unless TestMain ends nicely.

https://stackoverflow.com/a/56591384

[nhatthm]

Just wondering if your tests are panicking. Go test coverage will not generate unless TestMain ends nicely.

stackoverflow.com/a/56591384

In my case, no panic