Problems in using teamcity to test dgraph

Users Dgraph
1

Using teamcity to test dgraph

Refer to the official test

What I Did

  1. After installing teamcity and testing other projects, there is no problem.
  2. Test dgraph
    These are my test steps,
        script {
            name = "Make Linux Build"
            scriptContent = "make GOOS=linux dgraph"
        }
        script {
            name = "Check protobuf"
            scriptContent = """
                cd ./protos
                go mod tidy
                make regenerate
                git diff --exit-code -- .
            """.trimIndent()
        }
        script {
            name = "Run unit tests"
            scriptContent = """
                #!/bin/bash
                
                if [ -f go.mod ]; then
                  export GO111MODULE=on
                fi
                
                # Run the Go test script. Or, run test.sh if the Go test script doesn't exist.
                if [ -d ./t ]; then
                  cd t; go build .
                  ./t -r
                  ./t
                else
                  echo "./t directory does not exist!"
                  exit 1
                fi
            """.trimIndent()
        }

My problems

Teamcity always fails to execute ./t, here is the log

11:52:23
  Found valid task: github.com/dgraph-io/dgraph/xidmap isCommon:true
11:52:23
  Running tests for 109 packages.
11:52:23
  Sent 1/109 packages for processing.
11:52:23
  Sent 2/109 packages for processing.
11:52:23
  Sent 3/109 packages for processing.
11:52:23
  Bringing up cluster test-tc-056-2...
11:52:23
  Bringing up cluster test-tc-056-1...
11:52:23
  Bringing up cluster for package: github.com/dgraph-io/dgraph/dgraph/cmd/alpha/mutations_mode
11:52:23
  Bringing up cluster test-tc-056-4...
11:53:00
  2021/01/14 11:53:00 While running command: "docker-compose -f ../dgraph/docker-compose.yml -p test-tc-056-2 up --force-recreate --remove-orphans --detach" Error: exit status 1
11:53:00
  Process exited with code 1
11:53:01
  Process exited with code 1 (Step: Run unit tests (Command Line))
11:53:00
  Step Run unit tests (Command Line) failed

But I always succeed when I execute

docker-compose -f ../dgraph/docker-compose.yml -p test-tc-056-2 up --force-recreate --remove-orphans --detach

in the working directory of teamcity on the server.

My teamcity environment

ubuntu 18.04 
go version go1.15.2 linux/amd64

Dgraph Metadata

dgraph version

20.11.0

2

Does your TeamCity agent user have the right permissions to run Docker and Docker Compose? Since your docker-compose command is failing during your build it sounds like there’s an issue running Docker as the agent user. It’d help if you shared any error logs along with the failed docker-compose command.

But I always succeed when I execute docker-compose ... in the working directory of teamcity on the server.

I suspect this works because the user you logged in as has sufficient permissions to run Docker (e.g., your user is root or is in the Docker group).

3

Thank you for your reply, @dmai

I checked docker and docker-compose in the previous steps, and then I saw that the operation of deleting container was successful, but it still reported an error when executing docker-compose up

Docker version 19.03.8, build afacb8b7f0
docker-compose version 1.27.4, build 40524192

  Removed container /test-tc-972-4_alpha3_1 with error: <nil>
16:41:33
  Removed container /test-tc-972-4_alpha2_1 with error: <nil>
16:41:33
  Removed container /test-tc-972-4_alpha1_1 with error: <nil>
16:41:34
  Removed network: test-tc-972-3_default
16:41:35
  Removed network: test-tc-972-1_default
16:41:35
  Removed network: test-tc-972-4_default
4

I modified the runfatal method and got the cmd error output

func runFatal(cmd *exec.Cmd) {
	var stderr bytes.Buffer
	cmd.Stderr = &stderr
	if err := cmd.Run(); err != nil {
		log.Fatalf("While running command: %q Error: %v\n",
			strings.Join(cmd.Args, " "), stderr.String())
	}
}

17:44:43
  ERROR: for test-tc-709-4_alpha3_1  Cannot start service alpha3: OCI runtime create failed: container_linux.go:349: starting container process caused "exec: \"/gobin/dgraph\": stat /gobin/dgraph: no such file or directory": unknown
17:44:43
   [1A [2K
17:44:43
  Creating test-tc-709-4_alpha2_1 ... error
17:44:43
   [1B
17:44:43
  ERROR: for test-tc-709-4_alpha2_1  Cannot start service alpha2: OCI runtime create failed: container_linux.go:349: starting container process caused "exec: \"/gobin/dgraph\": stat /gobin/dgraph: no such file or directory": unknown
17:44:43
   [3A [2K
17:44:43
  Creating test-tc-709-4_alpha1_1 ... error
17:44:43
   [3B
17:44:43
  ERROR: for test-tc-709-4_alpha1_1  Cannot start service alpha1: OCI runtime create failed: container_linux.go:349: starting container process caused "exec: \"/gobin/dgraph\": stat /gobin/dgraph: no such file or directory": unknown
17:44:43
   [2A [2K
17:44:43
  Creating test-tc-709-4_zero1_1  ... error

Then I found that the $GOPATH environment variable was not fetched in teamcity, so the dgraph binary not being in the data volume.

Because the data volume configuration in docker compose is like this

    volumes:
    - type: bind
      source: $GOPATH/bin
      target: /gobin
      read_only: true