Orderasc/desc by object that is missing the field data

u007 · October 11, 2018, 8:42am

ive some old data that did not declare field created_at during my initial test.
later on, i’ve inserted those fields.
when i tried to sort by the missing field, the records only show those with this field declared. is this a norm?

i have not test with filter, what if i filter by this field with not equal?
will those old records without this field being enter also went missing?

as seen below, the result output will defer from count

query all($nothing: string){
  
users(func: eq(_type, "users") orderdesc: created_at)   {
		uid
		email
		created_at
		updated_at
}
  users_count(func: eq(_type, "users"))  {
	count(uid)
}
}

u007 · October 11, 2018, 10:34am

ive been testing delete,
and realize some how the data become inconsistant.
sometimes the list shows all records, but sometimes it shows only those records with created_at field specified

MichelDiz · October 11, 2018, 2:27pm

Hi James,

Not sure, but by logic, it seems like this would happen anyway. Because Dgraph can only Sort by what is indexed. You can create a task in your application to hunt those nodes that have no date and add a generic date.

Fill in an Issue about it.

Well I did some tests on:

Dgraph version   : v1.0.9
Commit SHA-1     : 22c13fce
Commit timestamp : 2018-10-02 16:45:53 -0700
Branch           : HEAD

{
  
users(func: eq(_type, "users"), orderdesc: created_at)   {
		uid
		email
		created_at
		updated_at
}
  users_count(func: eq(_type, "users"))  {
	count(uid)
}
}

Result:

{
  "data": {
    "users": [
      {
        "uid": "0x4",
        "email": "contact@edward.com",
        "created_at": "2018-01-01T00:00:00Z",
        "updated_at": "2019-01-01T00:00:00Z"
      },
      {
        "uid": "0x5",
        "email": "contact@edward.com",
        "created_at": "2018-01-01T00:00:00Z",
        "updated_at": "2019-01-01T00:00:00Z"
      },
      {
        "uid": "0x6",
        "email": "contact@edward.com",
        "created_at": "2018-01-01T00:00:00Z",
        "updated_at": "2019-01-01T00:00:00Z"
      },
      {
        "uid": "0x7",
        "email": "contact@edward.com",
        "created_at": "2018-01-01T00:00:00Z",
        "updated_at": "2019-01-01T00:00:00Z"
      }
    ],
    "users_count": [
      {
        "count": 8
      }
    ]
  },
  "extensions": {
    "server_latency": {
      "processing_ns": 3002300
    },
    "txn": {
      "start_ts": 22
    }
  }
}

Mutation was:


{
   "set":[
      {
         "_type":"users",
         "name":"Edward",
         "email":"contact@edward.com",
         "created_at":"2018",
         "updated_at":"2019"
      },
      {
         "_type":"users",
         "name":"Edward",
         "email":"contact@edward.com",
         "created_at":"2018",
         "updated_at":"2019"
      },
      {
         "_type":"users",
         "name":"Edward",
         "email":"contact@edward.com",
         "created_at":"2018",
         "updated_at":"2019"
      },
      {
         "_type":"users",
         "name":"Edward",
         "email":"contact@edward.com",
         "created_at":"2018",
         "updated_at":"2019"
      },
      {
         "_type":"users",
         "name":"Edward",
         "email":"contact@edward.com"
      },
      {
         "_type":"users",
         "name":"Edward",
         "email":"contact@edward.com"
      },
      {
         "_type":"users",
         "name":"Edward",
         "email":"contact@edward.com"
      },
      {
         "_type":"users",
         "name":"Edward",
         "email":"contact@edward.com"
      }
   ]
}

u007 · October 12, 2018, 4:40am

hi @MichelDiz

okay thank you.
i will do the manual update for addition of new fields…

are there any mutation in dgraph based on certain filter(s) that set all nodes to the value i wish?
for example are those without created_at and set it to 0001-01-01 ?

thank you

MichelDiz · October 12, 2018, 1:24pm

Nops, but you could try to do this as an upsert procedure Get started with Dgraph
may work good. Use “has(created_at)” instead of “eq”.

If the searched Node has “created_at” it does nothing. If it does not, it returns the UID and you generate a mutation with it.

u007 · October 16, 2018, 8:52am

Hi @MichelDiz

okay noted, so ive to do mutation in application layer for all existing records.
but how do i query for those field without created_at?

MichelDiz · October 16, 2018, 3:30pm

Maybe something like this:

{
  
users(func: eq(_type, "users")) @filter(NOT has(created_at))   {
		uid
		email
		created_at #let it here just to ensure
		updated_at #let it here just to ensure
}

}

And then do a query for “updated_at” just in case.

u007 · October 16, 2018, 4:02pm

Awesome, thank you
i thought its !has but didnt work

u007 · October 17, 2018, 9:59am

hi @MichelDiz

i have this finding:
with this same query, i queries this multiple times on ratel ui,
and 50% of the time, it return results with actual data, but another 50% only return records with uid field (other records without created_at). and i could not delete those uid only records

query all($nothing: string){
	users(func: eq(_type, "users"), first: 10, orderdesc: created_at)   {
		uid
		email
		username
		password
		first_name
		last_name
		mobile_no
		office_no
		gender
		country
		confirmed_at
		locked_at
		deleted_at
		reset_token
		status
		signup_type
		online
		online_at
		ip_address
		created_at
		updated_at
		roles : with_role {
			uid
			name 
		}
		work_for : work_for {
			uid
			code
			name
			status
			country
			created_at
			updated_at 
			 
		}
	}

	users_count(func: eq(_type, "users"))  @filter(has(created_at)) {
		count(uid)
	}
}

ok result:

{
  "data": {
    "users": [
      {
        "uid": "0xea63",
        "email": "dsfsdf@example.com",
        "username": "dsfsdf@example.com",
        "password": "",
        "first_name": "dsfsd",
        "last_name": "dsfdsf",
        "mobile_no": "4",
        "office_no": "",
        "gender": "M",
        "country": "AD",
        "confirmed_at": "0001-01-01T00:00:00+06:55",
        "locked_at": "0001-01-01T00:00:00+06:55",
        "deleted_at": "0001-01-01T00:00:00+06:55",
        "reset_token": "",
        "status": 1,
        "signup_type": "email",
        "online": 0,
        "online_at": "0001-01-01T00:00:00+06:55",
        "ip_address": "",
        "created_at": "2018-10-16T09:24:48.24963+08:00",
        "updated_at": "2018-10-16T09:25:31.91972Z"
      },
      {
        "uid": "0xea62",
        "email": "ddsfsdf@example.com",
        "username": "ddsfsdf@example.com",
        "password": "",
        "first_name": "sdfdss",
        "last_name": "dfdsd",
        "mobile_no": "2324324",
        "office_no": "",
        "gender": "M",
        "country": "",
        "confirmed_at": "0001-01-01T00:00:00+06:55",
        "locked_at": "0001-01-01T00:00:00+06:55",
        "deleted_at": "0001-01-01T00:00:00+06:55",
        "reset_token": "",
        "status": 1,
        "signup_type": "email",
        "online": 0,
        "online_at": "0001-01-01T00:00:00+06:55",
        "ip_address": "",
        "created_at": "2018-10-15T03:00:36.077012+08:00",
        "updated_at": "2018-10-16T07:26:56.430786Z",
        "roles": [
          {
            "uid": "0xc369",
            "name": "superadmin"
          },
          {
            "uid": "0xc36b",
            "name": "manager"
          },
          {
            "uid": "0xc36c",
            "name": "staff"
          }
        ]
      },
      {
        "uid": "0x2720",
        "email": "test5@test.com",
        "username": "test5@test.com",
        "password": "",
        "first_name": "sdfsdf tan",
        "last_name": "sdfdsf sdfdsf",
        "mobile_no": "2423432",
        "office_no": "",
        "gender": "M",
        "country": "MY",
        "confirmed_at": "0001-01-01T00:00:00+06:55",
        "locked_at": "0001-01-01T00:00:00+06:55",
        "deleted_at": "0001-01-01T00:00:00+06:55",
        "reset_token": "",
        "status": 0,
        "signup_type": "email",
        "online": 0,
        "online_at": "0001-01-01T00:00:00+06:55",
        "ip_address": "",
        "created_at": "2018-09-14T09:03:48.808329+08:00",
        "updated_at": "2018-10-16T05:56:07.528755Z",
        "roles": [
          {
            "uid": "0xc36b",
            "name": "manager"
          }
        ]
      },
      {
        "uid": "0x271e",
        "email": "tester@tester.com",
        "username": "tester@tester.com",
        "password": "",
        "first_name": "sdfsdfsdfdsf",
        "last_name": "sdfdsfdsf",
        "mobile_no": "3432423432",
        "office_no": "",
        "gender": "F",
        "country": "MY",
        "confirmed_at": "0001-01-01T00:00:00+06:55",
        "locked_at": "0001-01-01T00:00:00+06:55",
        "deleted_at": "0001-01-01T00:00:00+06:55",
        "reset_token": "",
        "status": 0,
        "signup_type": "email",
        "online": 0,
        "online_at": "0001-01-01T00:00:00+06:55",
        "ip_address": "",
        "created_at": "0001-01-01T00:00:00+06:55",
        "updated_at": "2018-10-16T09:25:17.077452Z",
        "roles": [
          {
            "uid": "0xc36a",
            "name": "admin"
          }
        ]
      },
      {
        "uid": "0x271f",
        "email": "test@test.com",
        "username": "test@test.com",
        "password": "",
        "first_name": "cdscdscds",
        "last_name": "tan",
        "mobile_no": "4324324324324234534",
        "office_no": "",
        "gender": "M",
        "country": "MY",
        "confirmed_at": "0001-01-01T00:00:00+06:55",
        "locked_at": "0001-01-01T00:00:00+06:55",
        "deleted_at": "0001-01-01T00:00:00+06:55",
        "reset_token": "",
        "status": 0,
        "signup_type": "email",
        "online": 0,
        "online_at": "0001-01-01T00:00:00+06:55",
        "ip_address": "",
        "created_at": "0001-01-01T00:00:00+06:55",
        "updated_at": "2018-10-16T09:23:08.259463Z"
      }
    ],
    "users_count": [
      {
        "count": 5
      }
    ]
  },
  "extensions": {
    "server_latency": {
      "parsing_ns": 75100,
      "processing_ns": 12097800,
      "encoding_ns": 1925700
    },
    "txn": {
      "start_ts": 90490,
      "lin_read": {
        "ids": {
          "1": 4989
        }
      }
    }
  }
}

uid only result:

{
  "data": {
    "users": [
      {
        "uid": "0x2716"
      },
      {
        "uid": "0x2715"
      },
      {
        "uid": "0x2713"
      },
      {
        "uid": "0x2712"
      },
      {
        "uid": "0x271d"
      },
      {
        "uid": "0x2717"
      },
      {
        "uid": "0x271c"
      },
      {
        "uid": "0x271b"
      },
      {
        "uid": "0x2711"
      },
      {
        "uid": "0x2719"
      }
    ],
    "users_count": [
      {
        "count": 5
      }
    ]
  },
  "extensions": {
    "server_latency": {
      "parsing_ns": 303600,
      "processing_ns": 13727200,
      "encoding_ns": 1918800
    },
    "txn": {
      "start_ts": 90489,
      "lin_read": {
        "ids": {
          "1": 4988
        }
      }
    }
  }
}

calummoore · October 18, 2018, 8:29am

I’ve also been seeing non-deterministic results when sorted value is missing.

MichelDiz · October 18, 2018, 1:53pm

This seems a different case, can you fill up an issue with more details?

u007 · October 18, 2018, 4:41pm

ive created an issue here:

https://github.com/dgraph-io/dgraph/issues/2672

system · November 17, 2018, 4:41pm

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Groupby + Orderasc in Query Dgraph dgraph	11	730	October 26, 2021
Identical query execution has inconsistent response Dgraph status:accepted , kind:bug , area:querylang , ticket:created	5	911	September 10, 2020
Order Results Dgraph kind:question , dgraph , area:querylang	6	738	December 27, 2020
Expected behaviour for sort queries Dgraph discussion	16	2440	March 25, 2021
Inconsistent response of sort as a result of indexing of a predicate Dgraph	8	931	September 21, 2020

Orderasc/desc by object that is missing the field data

Related topics