hhtlxhhxy
January 18, 2019, 8:45am
1
Hi,
I0118 15:09:42.488943 44827 pool.go:215] Connection established with 10.237.7.231:5080
I0118 15:09:42.488998 44827 pool.go:215] Connection established with 10.237.7.230:7083
I0118 15:09:42.489107 44827 pool.go:215] Connection established with 10.237.7.232:7083
E0118 15:39:42.485876 44827 pool.go:204] Echo error from 10.237.7.231:5080. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0118 15:39:42.486024 44827 pool.go:204] Echo error from 10.237.7.230:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
I0118 15:39:42.491800 44827 pool.go:215] Connection established with 10.237.7.230:7083
I0118 15:39:42.491987 44827 pool.go:215] Connection established with 10.237.7.231:5080
W0118 16:07:11.615756 44827 node.go:352] No healthy connection to node Id: 3 addr: [10.237.7.232:7083], err: Unhealthy connection
E0118 16:07:11.616109 44827 pool.go:204] Echo error from 10.237.7.232:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
I0118 16:07:11.618927 44827 pool.go:215] Connection established with 10.237.7.232:7083
E0118 16:24:51.542813 44827 pool.go:204] Echo error from 10.237.7.232:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
I0118 16:24:52.004703 44827 pool.go:215] Connection established with 10.237.7.232:7083
E0118 16:24:52.547590 44827 draft.go:443] Lastcommit 39802 > current 28792. This would cause some commits to be lost.
E0118 16:27:12.542670 44827 pool.go:204] Echo error from 10.237.7.232:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0118 16:27:12.542820 44827 pool.go:204] Echo error from 10.237.7.231:5080. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
W0118 16:27:12.542830 44827 node.go:352] No healthy connection to node Id: 3 addr: [10.237.7.232:7083], err: Unhealthy connection
E0118 16:27:12.543024 44827 pool.go:204] Echo error from 10.237.7.230:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
I0118 16:27:12.544722 44827 pool.go:215] Connection established with 10.237.7.232:7083
I0118 16:27:12.544911 44827 pool.go:215] Connection established with 10.237.7.231:5080
I0118 16:27:12.544950 44827 pool.go:215] Connection established with 10.237.7.230:7083
And the error make the RAFT between alpha nodes unstable, I think the timeout is short, the code
func (p *Pool) UpdateHealthStatus(printError bool) error {
conn := p.Get()
query := new(api.Payload)
query.Data = make([]byte, 10)
x.Check2(rand.Read(query.Data))
c := pb.NewRaftClient(conn)
// Ensure that we have a timeout here, otherwise a network partition could
// end up causing this RPC to get stuck forever.
ctx, cancel := context.WithTimeout(context.Background(), time.Second)
defer cancel()
resp, err := c.Echo(ctx, query)
if err == nil {
x.AssertTruef(bytes.Equal(resp.Data, query.Data),
"non-matching Echo response value from %v", p.Addr)
p.Lock()
p.lastEcho = time.Now()
p.Unlock()
} else if printError {
1s , is it too short?And if I can config the timeout by myself, I think it is cool.
mrjn
January 19, 2019, 1:17am
2
1s should be sufficient to Echo between nodes. How far are these machines – halfway across the world? Are you sure these network errors are not due to some config issues?
hhtlxhhxy
January 21, 2019, 2:25am
3
The three alpha nodes are under the same switch, the ping delay between them is less than 0.1ms.
Is there something wrong whit the config of my dgraph cluster?
Zero:
dgraph zero --my 10.237.7.231:5080 --replicas 3 --log_dir /export/data/dgraph
Alpha:
dgraph alpha --debugmode --pending_proposals 20 --max_retries 10 --my 10.237.7.231:7083 --lru_mb 163840 --zero 10.237.7.231:5080 -o 3 --log_dir /export/data/dgraph
dgraph alpha --debugmode --pending_proposals 20 --max_retries 10 --my 10.237.7.230:7083 --lru_mb 163840 --zero 10.237.7.231:5080 -o 3 --log_dir /export/data/dgraph
dgraph alpha --debugmode --pending_proposals 20 --max_retries 10 --my 10.237.7.232:7083 --lru_mb 163840 --zero 10.237.7.231:5080 -o 3 --log_dir /export/data/dgraph
mrjn
January 21, 2019, 2:34am
4
The config looks ok. Can you paste the full logs?
hhtlxhhxy
January 21, 2019, 2:44am
5
91E\ì½Ù%g&ÆnIк@Ò\0º*:ÿ}îjKVgXHvÕDádÄaf4cÉLNcz=Ýé^£+½.ôzÙç~Îÿ{¸ÓOØQlIÕÍÌ<aæþ/¶ÛgÝ¿ß¾ûõn{¶»ù°ûÓÙëíÛ7Ýöâímwqýú½'ùRJç6ïÅþ«þ©ö_&ú¶Îhcéïf£tpÖ¼·y÷4ìüëþön{Còf÷f{~·½Ò^eSþüÉN×çg¿ÐAYZÿÌû:nîék}²VÅð¾UË»óËÝ/´wIY|ÚEúu6;|÷tü]çc§MR!hÓwûà»Fw9¤`¼1þ}c7_~òëÏ>ÿ·ÝÙî;z:ƨÑÇçW׿0V;CüÞ§GÝ|uq~õí/ôû?õâýà«{¶[_¿÷ÿ ¿ÿ×ÿéëÿþéúÿûáþGk'÷ßßÛ¨§~îëÿç÷ÿ½ÿì_üçïýå{ïýf{ºùݳÙá³÷þúóÞ{ñ¿Òôï¿ø¿#ùÑW_}1üÿÿþü¯&?òÃçïýï½÷/O¯/»íÛ·»îrw·=ÛÞmO¾ýͧÝí.ÿøfw³ûåÍõå-hýÏô¿~ï½ÿ¦þüßßoo¶WwçW»÷^½½8¿½Sêù¿üÓ?¿sw÷öäÃïßvÛo»¿;½¸¾?ëè÷>|÷á©qÁ½>Õö̦÷çÿ#¥?øB>k>øWÖþê_ºÃÁ°¸Ýg¿ýåï:£tVZÇ´6Q«NÛCþÓ?ïÇ÷ýÿã~þb¿,ÿÉdþ·¿ÿP©¤ÿÊ:oÕ«í_}òVc÷Wÿѧ:ü7îÓÜûékã?ø4}S~¬?ýô¸½ªÚÿù¤Àþ·~¦ÿÿ¤ÿÿ_O®ÿmt]¶Áõþwióùgôiõ?¼èNovô³×W¨¢fR ßñx{A+gÿÉ^8æë¹u?¾õ?É*Lpqrÿ]æôÿãëóë×oÎ/vþöíÎèºl ß>TúCZx¢Õû_Ü_]_½Þ\_m.·§oHçlxùþÇçWÛN6ß_Üm¾?¿{³y}ºy};Mÿ·ùæúfCòþÝÛ˳àÞÇ#пwøÆ%ØýÙ_üòåååÙÙæÍËËÛÛî¾ÿÚܽ¡Ç<;?ëù¿õrsyûúýÏ ËÃv$hT°Í 7çWçwÝëë^nÞÿÓþÈo¾ÛÝÜ´![çdóîHnNíû\_^ßm¾üõGèÞ:Ùlµ
Ùú³Ã÷ ¡èØ\¾ÝkõvN\>Ñ~óJJ½ÿ1Ù&§o6õëdóë}úþ¯®G¼Ïkóþû¿¤ÅÙ?àõ7ßo/6gק÷»«»^:þ|óÝù-=L[²5è·Ýþ_w=³óÛÓû[p¸Ýl_]ßßHâkF`øá1¯®7·Û6oÎ7w×´ät8è½ïiØ?ñÆð»·ÛÓoaïtï¿ÿùùéîêÔwÛóëûÛ6÷Wdgôô>zKGh·ùýý«óÓÍþG7¦#mquvxØO
Óýt´þo¸9ýæËî? ÿ¤ÃO¾}uûóÍgW§Ä{~<©³rnw7´8Z/www8ÙoÏ^ãù¶¯è:\\oÏ6×o±â'ËËí[æèh4£ùÝöâ4_G7%®§gýÝÛ]¿¿9¿Û}°}C¡';ðùôãá¶
ôoO6ÿðéùÍÉ÷?ýð×/¸:ý#~ùöäîé+¼íçôPDõ7×g»=ü4]Êñ§fóÛûË?§öö«ë½Û½¥üÍö]ÿû_ÿ»ÝI K8¦Þ|¾ûnw~sqwNvùî^?Üçö$,¾¢»|ûæúâì$xO"8üfwÙ/ÿíÇ?ûÿïw7×_-øåÝöâôûÏw5< ÉYï|(oóËóýsjò7£Ó¬Ã÷èÑ^\ÝÝÝKobC§éÝõÍ-½üþ«ß]}rq}»Öîr{µ}½;ûê-õ7Û[|òîãíÝéO®ï¯îNTùwÏXm¾ íúÝÕÅûþêæþêî@¶ðD½ÓßäWæLýûÙÉÈÞ¨zâ®h_z¡f_n>º¸Ø¨áäÞÒîï®èÞ_mÔíFP=µ
7!MÒïéoѵ£U½ o¯oAÿöG¯ÀÛzÞ®_sô0ÚE,§ºZ÷è*ü㼧Fbr|hý74ßõ7²Ý
Ûé¤'7#t>X´»?J0tÊ ÁÜåã Fûãs§Ò 1ôÚjþsúqT«M¾.eÎR$_p¤')LH¦|2Húµ§Löþ!I«i!r$ÓÊS»©&ïm]ç}δ©ìáQk-e]ɹpíÊ{HÆßä½¢÷övá½Mð+g\ͳ´áäcdV+$éÞèÉZº@OiæµÕ+ÇÙî¸Ü=¿;Ö®Q¤«£'ÛÓÕq)êkKéèîLº÷]ò»ÀÌ+÷ÛùÎÚÉîxº;.ø
'wf$ÝãÆ$ÓÒ´F~ÃYyJOwGå I\ëæIºËãqyôd$aîM²wí)éò<<é Æ÷IE~{¼Ò+rènõt{TsZç5Îú&©ô("y¿FîO°¤yb°·ÜçÕçÌ$'4IùÕ»³ú¤´Nè²?/;_GDÔÓN&w»p×I9M´õá!Q{¤âÂ6Å5¡/¡
º5cr1óDÝê>"2~r HÂ/È¥W7TJ;OªÈG¥Í-W_ÊÈÅ QpÚ-Ó´fÃQRGq²¦¤"´@Ô¯¾>)$=½¤¤O1ò9)ÒIzz¤òÂæU¢s è.êÈ_Ó¼¾¦¤T¬i¯b
üæõsÚk¦k¡30X¢aEÙéKxY%Ýd5üá'ofuû¡´PÏä$ñïOÞÚêV"áñ*i¨-]Õªk¦²!¥'
U4ü¢jµfàRQÚ¥ QºRdk^Li½v¨é('D
9÷ÖÙ¸°¦ôkDsy¹WRÙÏÛZÕ×fBÕÓ¡ÒvÁ%xåP^M©éÐ¥ÊH-ø^jåVè)'¾VÉ.¬1+®éU<+ü%ëÕP¥«½ú¬pôd]mê2»ia]M\sÃ.i
RU.[fÍú1ÐUSaEªL1òæy¢vÍ6ÐU÷;÷®ZúNÜú£õ'Ë
÷)8ºZTãêÍå&ËJÚ,Jy@;µº¤®¦÷´Ï:%^[i·0½º
%@ê*fÒôàü¸6ÐW&5«U½'²IKV7ÎTR¥³UGÀ«UÙ
5{VRX&ÚçGû5ez¥ÜCª½Â¢K°°[~Me[(¬ÊDIcCJk)^´r,4Ö$"kdP+À§JÓ'uclÔ/ì°¤¯¬<)"z1¤¼`¿º¢êùÉ£B_§²`YéV®ªíõ7ª¤¯|vK·*®é+Ûë+;9©_Õfk'ÕÎ+PuX×¼dE¿îõÕt]ɳ"-¸ä긦¯lïZ=0.jïZi¬Ñ͵¨|«É
@aÑn-¬´¾®½ÂìÖë3Kº5ùµd¯°bP%ï
.뺦5ûÊîÕdà^%R%«Õgâ*ùWP¡²%µêà`MäGÈÂ,lôªe¡±y°®º÷°B¶Kr Uªp±\P-h/dç¯×ÞÅ¥«SZ
u+½za¡°üäQ¡°èQn-쥦Da
&µà¶DWÒÅr~B.%½ÌV£×«¥&û¥]0½¯t¤²Lì\,ã]\H¬ºX®w±P¥keCÙVÄÊ"§qBq£rQÆèÁâz¥'ËJ*tO¼/@ÒquHe39PY9ÁiªÄ@5L¦ìÂn´z®h
§r+kµàc¢oõÆ
iB ÅìhvuYi U,+i,]JÛµlë5V®@ï®*1Hejz'Ï
຺õuƲ«åi|Ìq)Q¾æºÕ0I*,ç-ãÖ4Ʋúl5½å½¶KT×/|¬+` ±³KAAãÒ?à ±ôhIJúËøuÙõpYMµrY/ÝW¯Wî«GÚÊLV.Ù;KÊÅ»µô4V%ÈìäÍVC*rhÚl¦O\°$üð¤±&ÆÙMè jè¬tT}ïºyÞ4aÍôå&TSM¡VÖ·4qE
d!vcÂZ¤ÙNÖäY¾±AAÖAE·}B5" l¶¸ñ½óCªd`/8¯&®Bt°Sª¨¦ MY8q-íûÖD°@cù°Tcâ}é{eíjîfáĵlïÓKЧ°Hµ,,@ZË
û!'ToOq!+lÒZ-ï}¬©neYÒ)^XøX"Ö(a¥Ì̬,¨e$fÔLvÚ»·ÛPðØWël§UÚâq¿¹ÝÝl¬wÚZe¨¢äaêæîúú[ú}ÓÑþÿaöXV)5
Ô¿¾¹¾{Y¿Ü|rs³»ºÛ|±ýænó=ÿº×£B¸ï¯o¾ê8ýúûmм7ÔsÒmÏÎnv··'¯ON^D,C2°û+Уsóróúß²¯Ý µãnwÖm6ú×W·×7w¼@vã-dÓËͯ¿úê÷ÇMYçBåºÞ^__ôe¬è~ò»ßþöÅ'_½ø¥ödýØEú/YY*ÍÒ¥GUfu'Èý!Z×WW»S4qþ{·»¹î6ÝÞ¿Æáë?<Ùè9]rõõãP±;ãÈéiÛ#ߥDËÈ<»¡RNÍoéo>ûÎìPíû+¼þ=³HåüJi¯kirOïó×ôd'ÿöݼÝínnO¾~¹¹Ømo®öqA+|~}õÕùé·¨Ýü¾y÷j·½>Ù|yw}³}½;QïN:gÁªÍGoQú{vÒWþ¢þõ÷»ßܾ&.%O?»úæ
ôñí ²¼Ù~ûßÝ_ßÜ_îKd³ûÃõ¡BöP>û»¡ZÍJf7ßBòüþæúíõíöâ×7ßooPÎ<û÷Ó¥ò6:Fì_¼AïÎýÕý¹j^Åùiú1ÚÿíÅÖû³ÛßÓÒÎ7Q`ü»áåæÓë«Ý°×ÃooNCÂ,ÝHÂ拉@$ܨcD"©¯nzLÇ÷ï®I÷'¤cHxîàæìêü+úýO.îI|Ül¾;ßâÑî_²ØÍ(T6¢Æ«4¦Ï.)ÿÅn^íN·X§ëïÉd¢s9íD%G^U_í¾ïUÝ|=\¤Í×/Þ!ñó¾£èü®ÿëv9ð÷-ªùÏvïÊ¿ö¿ñrÎÚ©È[4Ë/0ßnçô¨´e$Pèåýùí]g?0¿lFNpýåé¬üêºh2XÛ
õù}æD=¬VâQÑ+~ Ô«K,ôærwùÖøÍù[RNô¬ôÀ§»óïh n®/çÌægÖ=¨5q3
fBO]w7·wÐw»ÍýÛ3üÑ3À«ÐÞ_Ñ2$¯TØÓ8ùëow?Xìû>Nþzÿ OÉÑÃÏÿ©ÿ+íäg³«ð³ÃêõãOÃC8÷ðÈþþ÷q1?ÊEí¹ÌHØ û£$ÌDo÷ÝíIüìOoovgçèpøÓÏäFÊw'?9'´ïµë·Û^¬Pÿø"É·ÛÛ[²ÎΣXzàý-=9äøç¾;?òûüýÿ`±Ç9k¸!ÏZ"г³½ÝÒöè¡¿zwõÕ¡oþ ¡÷É)ÿIõ
iÅò«íîÜ«ò©Ëlõ+O.ûmf·±O5.ȶ¯Húôæp-·²!Bg÷öLA¼!£½\08P0s
show original
The alpha node in 10.237.7.232.
hhtlxhhxy
January 22, 2019, 3:04am
6
Hello mrjn,
New binary:
[hehaitao3@A01-R06-I7-232-J33GKR6 dgraph]$ dgraph-new version
Dgraph version : v1.0.12-rc6
Commit SHA-1 :
Commit timestamp :
Branch : hehaitao/fix_health_timeout
Go version : go1.11.1
For Dgraph official documentation, visit https://docs.dgraph.io.
For discussions about Dgraph , visit http://discuss.hypermode.com.
To say hi to the community , visit https://dgraph.slack.com.
Licensed variously under the Apache Public License 2.0 and Dgraph Community License.
Copyright 2015-2018 Dgraph Labs, Inc.
New binary log:
[hehaitao3@A01-R06-I7-232-J33GKR6 dgraph]$ cat dgraph-new.ERROR|grep DeadlineExceeded
E0121 16:17:10.281438 77853 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0121 16:17:40.759539 77853 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0121 16:20:45.759165 77853 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0121 16:22:04.759190 77853 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0121 17:50:00.766776 77853 pool.go:204] Echo error from 10.237.7.230:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0121 19:40:27.759302 77853 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0121 20:13:59.759299 77853 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0121 20:14:00.755302 77853 pool.go:204] Echo error from 10.237.7.231:5080. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0121 20:14:01.484338 77853 groups.go:636] While sending membership update: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0121 20:14:21.759190 77853 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0121 23:05:20.759164 77853 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0122 02:36:48.486784 77853 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0122 02:36:48.486971 77853 pool.go:204] Echo error from 10.237.7.230:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0122 02:37:21.086037 77853 pool.go:204] Echo error from 10.237.7.231:5080. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0122 02:44:51.759674 77853 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0122 04:20:34.310761 77853 pool.go:204] Echo error from 10.237.7.230:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0122 04:21:22.759439 77853 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0122 04:25:04.759165 77853 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0122 05:19:44.759274 77853 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0122 05:49:06.759306 77853 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0122 08:59:11.759255 77853 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0122 10:11:02.047545 77853 pool.go:204] Echo error from 10.237.7.230:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0122 10:24:12.759276 77853 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0122 10:34:52.759089 77853 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0122 10:35:14.759149 77853 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
[hehaitao3@A01-R06-I7-232-J33GKR6 dgraph]$ cat dgraph-new.ERROR|grep DeadlineExceeded|wc -l
25
[hehaitao3@A01-R06-I7-232-J33GKR6 dgraph]$
Old binary log:
E0120 16:16:08.463222 139969 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 16:17:01.463306 139969 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 16:17:47.463335 139969 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 16:54:13.473504 139969 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 16:54:21.447099 139969 pool.go:204] Echo error from 10.237.7.231:5080. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 16:54:21.463205 139969 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 16:54:31.463275 139969 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 16:54:37.463214 139969 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 16:54:39.463247 139969 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 16:56:49.463231 139969 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 17:20:00.246351 139969 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 17:20:02.447647 139969 pool.go:204] Echo error from 10.237.7.231:5080. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 17:20:02.678359 139969 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 17:20:07.534880 139969 pool.go:204] Echo error from 10.237.7.231:5080. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 17:20:07.534989 139969 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 17:20:07.535004 139969 pool.go:204] Echo error from 10.237.7.230:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 17:20:30.447056 139969 pool.go:204] Echo error from 10.237.7.231:5080. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 17:20:30.463259 139969 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 17:20:45.463190 139969 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 17:20:50.463186 139969 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 17:21:00.463169 139969 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 17:21:03.446990 139969 pool.go:204] Echo error from 10.237.7.231:5080. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 17:21:17.463237 139969 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 17:21:25.463191 139969 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 17:32:47.463244 139969 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 17:35:17.463309 139969 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 17:39:41.465065 139969 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 17:39:41.522029 139969 pool.go:204] Echo error from 10.237.7.231:5080. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 17:39:53.772889 139969 pool.go:204] Echo error from 10.237.7.230:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 17:39:53.773279 139969 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 17:39:57.442494 139969 pool.go:204] Echo error from 10.237.7.231:5080. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 17:40:06.014029 139969 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 17:40:10.127721 139969 pool.go:204] Echo error from 10.237.7.230:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 17:40:15.176120 139969 pool.go:204] Echo error from 10.237.7.230:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 17:40:15.176218 139969 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 17:40:15.176274 139969 pool.go:204] Echo error from 10.237.7.231:5080. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 17:56:37.463259 139969 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 18:03:39.463319 139969 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 18:03:52.463288 139969 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 18:03:59.463277 139969 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 18:08:35.463213 139969 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 18:09:21.463211 139969 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 18:10:17.463289 139969 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 18:34:07.842982 139969 pool.go:204] Echo error from 10.237.7.231:5080. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 18:34:10.276296 139969 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 18:34:10.276368 139969 pool.go:204] Echo error from 10.237.7.230:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 18:39:03.090659 139969 pool.go:204] Echo error from 10.237.7.230:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 18:39:12.639953 139969 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 18:39:12.639959 139969 pool.go:204] Echo error from 10.237.7.231:5080. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 18:39:39.463441 139969 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 18:40:10.464185 139969 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 18:40:26.463371 139969 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 18:41:43.463216 139969 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 18:41:59.463237 139969 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 18:42:03.463336 139969 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 18:43:52.707130 139969 pool.go:204] Echo error from 10.237.7.230:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 18:43:52.707286 139969 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 18:51:45.463244 139969 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 18:51:49.463336 139969 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 18:51:59.463294 139969 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 19:04:09.463361 139969 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 19:04:14.701861 139969 pool.go:204] Echo error from 10.237.7.230:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 19:04:28.146527 139969 pool.go:204] Echo error from 10.237.7.231:5080. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 19:06:41.463294 139969 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 19:07:47.463307 139969 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 19:21:39.463226 139969 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 19:21:45.463310 139969 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 19:22:07.463310 139969 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 19:22:29.463248 139969 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 19:22:33.463252 139969 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 19:29:35.643046 139969 pool.go:204] Echo error from 10.237.7.231:5080. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 19:29:41.358028 139969 pool.go:204] Echo error from 10.237.7.230:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 19:29:41.482968 139969 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 19:44:54.463142 139969 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 19:45:07.463275 139969 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 19:47:04.463499 139969 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 19:47:28.463291 139969 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 19:55:36.804218 139969 pool.go:204] Echo error from 10.237.7.230:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 19:55:36.804288 139969 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 19:55:36.804446 139969 pool.go:204] Echo error from 10.237.7.231:5080. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 20:16:53.463321 139969 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 20:17:01.463232 139969 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 20:19:16.715622 139969 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 20:19:25.151088 139969 pool.go:204] Echo error from 10.237.7.231:5080. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 20:19:25.151105 139969 pool.go:204] Echo error from 10.237.7.230:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 20:19:25.151458 139969 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 20:19:44.281587 139969 pool.go:204] Echo error from 10.237.7.231:5080. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 20:19:44.281668 139969 pool.go:204] Echo error from 10.237.7.230:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 20:19:44.281844 139969 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 20:27:22.463238 139969 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 20:27:31.463223 139969 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0120 20:28:06.463175 139969 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code =
I captured a small portion of the error logs. At the same time range, the number of the error log is 396. And the new binary is 25.
mrjn
January 22, 2019, 3:11am
7
The main question is, why are there connection errors anyways? You said, these machines are on the same rack.
Can you run a ping from one of these alphas to other servers, and paste that log as well (ensure that the timestamps are in the same timezone and match)? That way, we can try to determine if this is a connection issue, or is this something specific to Dgraph?
hhtlxhhxy
January 22, 2019, 5:13am
8
the alpha error log:
E0122 12:14:10.083323 76857 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0122 12:14:18.083198 76857 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0122 12:14:28.083152 76857 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0122 12:14:35.419328 76857 groups.go:869] Error in oracle delta stream. Error: rpc error: code = Canceled desc = context canceled
E0122 12:14:41.083193 76857 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0122 12:14:48.083247 76857 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0122 12:14:58.389756 76857 groups.go:869] Error in oracle delta stream. Error: rpc error: code = Canceled desc = context canceled
E0122 12:21:17.083762 76857 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0122 12:22:28.189989 76857 groups.go:869] Error in oracle delta stream. Error: rpc error: code = Canceled desc = context canceled
E0122 12:22:28.189797 76857 pool.go:204] Echo error from 10.237.7.231:5080. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0122 12:22:29.352068 76857 pool.go:204] Echo error from 10.237.7.232:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0122 12:27:56.728473 76857 groups.go:869] Error in oracle delta stream. Error: rpc error: code = Canceled desc = context canceled
E0122 12:28:25.083196 76857 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0122 12:37:59.083484 76857 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0122 12:45:55.083246 76857 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0122 12:58:25.083267 76857 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0122 13:02:27.083230 76857 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0122 13:02:29.083274 76857 pool.go:204] Echo error from 10.237.7.231:7083. Err: rpc error: code = DeadlineExceeded desc = context deadline exceeded
the ping log:
64 bytes from 10.237.7.231: icmp_seq=2976 ttl=64 time=0.088 ms 2019-01-22 12:14:00
64 bytes from 10.237.7.231: icmp_seq=2977 ttl=64 time=0.053 ms 2019-01-22 12:14:01
64 bytes from 10.237.7.231: icmp_seq=2978 ttl=64 time=0.090 ms 2019-01-22 12:14:02
64 bytes from 10.237.7.231: icmp_seq=2979 ttl=64 time=0.107 ms 2019-01-22 12:14:03
64 bytes from 10.237.7.231: icmp_seq=2980 ttl=64 time=0.067 ms 2019-01-22 12:14:04
64 bytes from 10.237.7.231: icmp_seq=2981 ttl=64 time=0.056 ms 2019-01-22 12:14:05
64 bytes from 10.237.7.231: icmp_seq=2982 ttl=64 time=0.055 ms 2019-01-22 12:14:06
64 bytes from 10.237.7.231: icmp_seq=2983 ttl=64 time=0.089 ms 2019-01-22 12:14:07
64 bytes from 10.237.7.231: icmp_seq=2984 ttl=64 time=0.062 ms 2019-01-22 12:14:08
64 bytes from 10.237.7.231: icmp_seq=2985 ttl=64 time=0.056 ms 2019-01-22 12:14:09
64 bytes from 10.237.7.231: icmp_seq=2986 ttl=64 time=0.099 ms 2019-01-22 12:14:10
64 bytes from 10.237.7.231: icmp_seq=2987 ttl=64 time=0.128 ms 2019-01-22 12:14:11
64 bytes from 10.237.7.231: icmp_seq=2988 ttl=64 time=0.100 ms 2019-01-22 12:14:12
64 bytes from 10.237.7.231: icmp_seq=2989 ttl=64 time=0.093 ms 2019-01-22 12:14:13
64 bytes from 10.237.7.231: icmp_seq=2990 ttl=64 time=0.115 ms 2019-01-22 12:14:14
64 bytes from 10.237.7.231: icmp_seq=2991 ttl=64 time=0.057 ms 2019-01-22 12:14:15
64 bytes from 10.237.7.231: icmp_seq=2992 ttl=64 time=0.066 ms 2019-01-22 12:14:16
64 bytes from 10.237.7.231: icmp_seq=2993 ttl=64 time=0.065 ms 2019-01-22 12:14:17
64 bytes from 10.237.7.231: icmp_seq=2994 ttl=64 time=0.072 ms 2019-01-22 12:14:18
64 bytes from 10.237.7.231: icmp_seq=2995 ttl=64 time=0.052 ms 2019-01-22 12:14:19
64 bytes from 10.237.7.231: icmp_seq=2996 ttl=64 time=0.054 ms 2019-01-22 12:14:20
64 bytes from 10.237.7.231: icmp_seq=2997 ttl=64 time=0.068 ms 2019-01-22 12:14:21
64 bytes from 10.237.7.231: icmp_seq=2998 ttl=64 time=0.068 ms 2019-01-22 12:14:22
64 bytes from 10.237.7.231: icmp_seq=2999 ttl=64 time=0.075 ms 2019-01-22 12:14:23
64 bytes from 10.237.7.231: icmp_seq=3000 ttl=64 time=0.063 ms 2019-01-22 12:14:24
64 bytes from 10.237.7.231: icmp_seq=3001 ttl=64 time=0.085 ms 2019-01-22 12:14:25
64 bytes from 10.237.7.231: icmp_seq=3002 ttl=64 time=0.061 ms 2019-01-22 12:14:26
64 bytes from 10.237.7.231: icmp_seq=3003 ttl=64 time=0.061 ms 2019-01-22 12:14:27
64 bytes from 10.237.7.231: icmp_seq=3004 ttl=64 time=0.122 ms 2019-01-22 12:14:28
64 bytes from 10.237.7.231: icmp_seq=3005 ttl=64 time=0.047 ms 2019-01-22 12:14:29
64 bytes from 10.237.7.231: icmp_seq=3006 ttl=64 time=0.047 ms 2019-01-22 12:14:30
64 bytes from 10.237.7.231: icmp_seq=3007 ttl=64 time=0.055 ms 2019-01-22 12:14:31
64 bytes from 10.237.7.231: icmp_seq=3008 ttl=64 time=0.080 ms 2019-01-22 12:14:32
64 bytes from 10.237.7.231: icmp_seq=3009 ttl=64 time=0.087 ms 2019-01-22 12:14:33
64 bytes from 10.237.7.231: icmp_seq=3010 ttl=64 time=0.058 ms 2019-01-22 12:14:34
64 bytes from 10.237.7.231: icmp_seq=3011 ttl=64 time=0.071 ms 2019-01-22 12:14:35
64 bytes from 10.237.7.231: icmp_seq=3012 ttl=64 time=0.049 ms 2019-01-22 12:14:36
64 bytes from 10.237.7.231: icmp_seq=3013 ttl=64 time=0.056 ms 2019-01-22 12:14:37
64 bytes from 10.237.7.231: icmp_seq=3014 ttl=64 time=0.062 ms 2019-01-22 12:14:38
64 bytes from 10.237.7.231: icmp_seq=3015 ttl=64 time=0.069 ms 2019-01-22 12:14:39
64 bytes from 10.237.7.231: icmp_seq=3016 ttl=64 time=0.118 ms 2019-01-22 12:14:40
64 bytes from 10.237.7.231: icmp_seq=3017 ttl=64 time=0.067 ms 2019-01-22 12:14:41
64 bytes from 10.237.7.231: icmp_seq=3018 ttl=64 time=0.070 ms 2019-01-22 12:14:42
64 bytes from 10.237.7.231: icmp_seq=3019 ttl=64 time=0.077 ms 2019-01-22 12:14:43
64 bytes from 10.237.7.231: icmp_seq=3020 ttl=64 time=0.104 ms 2019-01-22 12:14:44
64 bytes from 10.237.7.231: icmp_seq=3021 ttl=64 time=0.069 ms 2019-01-22 12:14:45
64 bytes from 10.237.7.231: icmp_seq=3022 ttl=64 time=0.071 ms 2019-01-22 12:14:46
64 bytes from 10.237.7.231: icmp_seq=3023 ttl=64 time=0.063 ms 2019-01-22 12:14:47
64 bytes from 10.237.7.231: icmp_seq=3024 ttl=64 time=0.050 ms 2019-01-22 12:14:48
64 bytes from 10.237.7.231: icmp_seq=3025 ttl=64 time=0.055 ms 2019-01-22 12:14:49
64 bytes from 10.237.7.231: icmp_seq=3026 ttl=64 time=0.059 ms 2019-01-22 12:14:50
64 bytes from 10.237.7.231: icmp_seq=3027 ttl=64 time=0.063 ms 2019-01-22 12:14:51
64 bytes from 10.237.7.231: icmp_seq=3028 ttl=64 time=0.107 ms 2019-01-22 12:14:52
64 bytes from 10.237.7.231: icmp_seq=3029 ttl=64 time=0.055 ms 2019-01-22 12:14:53
64 bytes from 10.237.7.231: icmp_seq=3030 ttl=64 time=0.049 ms 2019-01-22 12:14:54
64 bytes from 10.237.7.231: icmp_seq=3031 ttl=64 time=0.054 ms 2019-01-22 12:14:55
64 bytes from 10.237.7.231: icmp_seq=3032 ttl=64 time=0.068 ms 2019-01-22 12:14:56
64 bytes from 10.237.7.231: icmp_seq=3033 ttl=64 time=0.074 ms 2019-01-22 12:14:57
64 bytes from 10.237.7.231: icmp_seq=3034 ttl=64 time=0.061 ms 2019-01-22 12:14:58
64 bytes from 10.237.7.231: icmp_seq=3035 ttl=64 time=0.059 ms 2019-01-22 12:14:59
64 bytes from 10.237.7.231: icmp_seq=3036 ttl=64 time=0.071 ms 2019-01-22 12:15:00
64 bytes from 10.237.7.231: icmp_seq=3037 ttl=64 time=0.065 ms 2019-01-22 12:15:01
64 bytes from 10.237.7.231: icmp_seq=3038 ttl=64 time=0.126 ms 2019-01-22 12:15:02
64 bytes from 10.237.7.231: icmp_seq=3476 ttl=64 time=0.098 ms 2019-01-22 12:22:20
64 bytes from 10.237.7.231: icmp_seq=3477 ttl=64 time=0.109 ms 2019-01-22 12:22:21
64 bytes from 10.237.7.231: icmp_seq=3478 ttl=64 time=0.115 ms 2019-01-22 12:22:22
64 bytes from 10.237.7.231: icmp_seq=3479 ttl=64 time=0.098 ms 2019-01-22 12:22:23
64 bytes from 10.237.7.231: icmp_seq=3480 ttl=64 time=0.144 ms 2019-01-22 12:22:24
64 bytes from 10.237.7.231: icmp_seq=3481 ttl=64 time=0.092 ms 2019-01-22 12:22:25
64 bytes from 10.237.7.231: icmp_seq=3482 ttl=64 time=0.105 ms 2019-01-22 12:22:26
64 bytes from 10.237.7.231: icmp_seq=3483 ttl=64 time=0.096 ms 2019-01-22 12:22:27
64 bytes from 10.237.7.231: icmp_seq=3484 ttl=64 time=0.119 ms 2019-01-22 12:22:28
64 bytes from 10.237.7.231: icmp_seq=3485 ttl=64 time=0.100 ms 2019-01-22 12:22:29
64 bytes from 10.237.7.231: icmp_seq=3486 ttl=64 time=0.087 ms 2019-01-22 12:22:30
64 bytes from 10.237.7.231: icmp_seq=3487 ttl=64 time=0.090 ms 2019-01-22 12:22:31
64 bytes from 10.237.7.231: icmp_seq=3488 ttl=64 time=0.115 ms 2019-01-22 12:22:32
64 bytes from 10.237.7.231: icmp_seq=3489 ttl=64 time=0.092 ms 2019-01-22 12:22:33
64 bytes from 10.237.7.231: icmp_seq=3490 ttl=64 time=0.104 ms 2019-01-22 12:22:34
64 bytes from 10.237.7.231: icmp_seq=3491 ttl=64 time=0.095 ms 2019-01-22 12:22:35
there are no packet loss
mrjn
January 22, 2019, 5:19am
9
Wao. That looks like an issue with Grpc. Can you file a bug against Dgraph, and we’ll try and investigate why are Echos timing out?
hhtlxhhxy
January 22, 2019, 5:27am
11
system
February 21, 2019, 5:27am
12
This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.