Error importing data from v1.0.8 to Slash GraphQL backend

Issues Dgraph Cloud / Slash GraphQL
1

In 2018 I prototyped a mindmapping app using self hosted dgraph v1.0.8 in a kubernetes cluster to store user content. I’m decommissioning the app for a period of time, intending to relaunch in the near future using Slash GraphQL instead of running my own dgraph cluster. I’ve exported my schema and a dump of all my content as a rdf.gz file. I was trying to import this data into my Slash GraphQL backend and running into some errors. I’m looking for help with how to get my exported data imported into Slash GraphQL to ensure that I’ve not lost any data before I decommission my old cluster.

I obtained the data export by kubectl execing into my 1 dgraph server node and running curl localhost:8080/admin/export. then I used kubectl cp ... to copy the .rdf.gz and .schema.gz files out of the kubernetes cluster.

Following the instructions here, I attempted to upload my data to my Slash GraphQL endpoint using the following docker container run:

docker run -it --rm \
  -v `pwd`/dgraph-1-2020-11-08-22-28.rdf.gz:/tmp/dgraph-1-2020-11-08-22-28.rdf.gz \
  -v `pwd`/dgraph-1-2020-11-08-22-28.schema:/tmp/dgraph-1-2020-11-08-22-28.schema \
  dgraph/dgraph:v20.07-slash \
  dgraph live --slash_grpc_endpoint=... \
  -f /tmp/dgraph-1-2020-11-08-22-28.rdf.gz \
  -s /tmp/dgraph-1-2020-11-08-22-28.schema \
  -t "<REDACTED>" -v 10

and get this error output

[Decoder]: Using assembly version of decoder
I1109 15:10:49.120563       1 init.go:102] 

Dgraph version   : v20.07.1-rc1-29-g43c04cff
Dgraph codename  : shuri-1
Dgraph SHA-256   : 2c55bb97c89eda3ca6a3cfae3508c1ca24892baff58ad4bb6df3f51726d99245
Commit SHA-1     : 43c04cff
Commit timestamp : 2020-10-28 11:35:44 +0530
Branch           : release/v20.07-slash
Go version       : go1.14.4

For Dgraph official documentation, visit ...
For discussions about Dgraph     , visit ...
To say hi to the community       , visit ...

Licensed variously under the Apache Public License 2.0 and Dgraph Community License.
Copyright 2015-2020 Dgraph Labs, Inc.


I1109 15:10:49.121274       1 util_ee.go:126] KeyReader instantiated of type <nil>

Running transaction with dgraph endpoint: xiagram.grpc.us-west-2.aws.cloud.dgraph.io:443

Processing schema file "/tmp/dgraph-1-2020-11-08-22-28.schema"
I1109 15:10:50.049245       1 xidmap.go:125] Assigned Uids: startId:720001 endId:730000 . Err: <nil>
Error while processing schema file "/tmp/dgraph-1-2020-11-08-22-28.schema": rpc error: code = Unknown desc = while lexing id:string @index(exact) .
edge:bool .
node:bool .
touid:uid @reverse .
~touid:uid .
content:string @index(exact,term) .
created:datetime @index(hour) @upsert .
creator:string @index(exact) .
deleted:bool .
fromuid:uid @reverse .
readers:[string] @index(exact) .
writers:[string] @index(exact) .
modified:datetime .
~fromuid:uid .
directionality:string .
 at line 5 column 0: Invalid schema. Unexpected ~
rpc error: code = Unknown desc = while lexing id:string @index(exact) .
edge:bool .
node:bool .
touid:uid @reverse .
~touid:uid .
content:string @index(exact,term) .
created:datetime @index(hour) @upsert .
creator:string @index(exact) .
deleted:bool .
fromuid:uid @reverse .
readers:[string] @index(exact) .
writers:[string] @index(exact) .
modified:datetime .
~fromuid:uid .
directionality:string .
 at line 5 column 0: Invalid schema. Unexpected ~

the .rdf file includes many lines like

...
<_:uid27ee> <~touid> _:uid27ef .
...

and I recall this property being important to the application code

2

You don’t need to specify the reverse edges in the schema, so you can remove edges that start with the tilde (~). You wouldn’t need them in your RDF file either as Dgraph’s reverse index maintains the reverse relationships automatically.

There are a number of changes from Dgraph v1.0.x to the latest Dgraph version that powers Slash GraphQL backends today. You can check out this doc page for the specific changes you’ll want to make in your schema and data: https://dgraph.io/docs/howto/migrate-dgraph-1-1. While the page is a how-to titled “Migrate to Dgraph v1.1”, it’s really about migrating from Dgraph v1.0 to later versions. We’ll update that title soon.

The changes you’ll want to note are:

  • In the schema, [uid] is for list uid types and uid is for singular (one-to-one) node types. So, you’ll want to update the uid predicates in your schema to [uid] unless you want to enforce one-to-one relationships.
  • Using the type system to define the outgoing predicates of your nodes to use with expand(_all_) queries.
  • HTTP API changes if you’re using that (with curl, for example)
1 Like
3

@dmai thanks very much for your detailed and lightning-fast reply! I will be looking into your response this weekend

1 Like
4

Let us know how the migration goes :slight_smile: Dgraph v1.0 sure does bring back memories!

5

Hey @peterwhitesell,

Did you get this to work eventually?

Tejas

6

Hi @gja, thanks for checking in. I actually haven’t got a chance to dig in yet. I’ll be sure to report back here when I do. hoping to carve out some time over this holiday week

7

Just following up here that after removing the explicit reverse edges starting with ~ from my schema and rdf file, I was able to use the dgraph live ... mechanism to import my data into Slash GraphQL. Thanks again for the help!