Cardano-db-sync: DB lookup fail in insertTxIn: tx hash (erikd/shelley-32byte-address-hash)

Which network is that? The only one that I know should work is @disassembler 's shelley_qa network.

this is shelley_qa

I will have a look tomorrow.

Okay thanks. Pls check too this:

[db-sync-node:Info:32] [2020-06-06 07:43:26.86 UTC] Shelley: No rollback required: db tip slot is 27908 ledger tip slot is 27908
[db-sync-node:Info:32] [2020-06-06 07:43:27.15 UTC] epochPluginInsertBlock: Inserting row in epoch table for epoch 21
[db-sync-node:Error:32] [2020-06-06 07:43:27.16 UTC] DB lookup fail in updateEpochNum.insertEpoch: epoch number 21
...
[db-sync-node:Error:78] [2020-06-06 07:42:37.72 UTC] DB lookup fail in updateEpochNum.insertEpoch: epoch number 22

[db-sync-node:Info:283] [2020-06-06 08:00:52.38 UTC] Starting chainSyncClient
[db-sync-node:Info:283] [2020-06-06 08:00:52.45 UTC] Cardano.Db tip is at slot 47627, block 2180
[db-sync-node:Info:288] [2020-06-06 08:00:52.45 UTC] Running DB thread
[db-sync-node:Info:288] [2020-06-06 08:00:52.61 UTC] Shelley: No rollback required: db tip slot is 47627 ledger tip slot is 47627
[db-sync-node:Error:288] [2020-06-06 08:00:53.11 UTC] DB lookup fail in insertTxIn: tx hash 54481fa557bd1872a2ce35271e3641cd619da2e31dd7287ce077b05ed7636f36
[db-sync-node:Info:288] [2020-06-06 08:00:53.11 UTC] Shutting down DB thread
[db-sync-node:Error:291] [2020-06-06 08:00:53.11 UTC] recvMsgRollForward: AsyncCancelled

• another start

[db-sync-node:Info:104] [2020-06-06 08:01:49.52 UTC] Cardano.Db tip is at slot 47874, block 2193
[db-sync-node:Info:109] [2020-06-06 08:01:49.52 UTC] Running DB thread
[db-sync-node:Info:109] [2020-06-06 08:01:49.78 UTC] Shelley: No rollback required: db tip slot is 47874 ledger tip slot is 47874
[db-sync-node:Error:109] [2020-06-06 08:01:50.26 UTC] DB lookup fail in insertTxIn: tx hash 78b4e35754058760705ea08c1234291e4c891e8435727778a1ef7e5a843e958d
[db-sync-node:Info:109] [2020-06-06 08:01:50.26 UTC] Shutting down DB thread
[db-sync-node:Error:112] [2020-06-06 08:01:50.29 UTC] recvMsgRollForward: AsyncCancelled

We have the same error on FF:

Connection to localhost 5432 port [tcp/postgresql] succeeded!
Started cardano-db-sync.service.
[db-sync-node:Info:4] [2020-06-06 11:25:32.78 UTC] NetworkMagic: RequiresMagic 42
[db-sync-node:Info:4] [2020-06-06 11:25:32.80 UTC] Initial genesis distribution populated. Hash 20472065206e2065207320692073202d20482061207320682020202020202020
[db-sync-node:Info:4] [2020-06-06 11:25:32.80 UTC] Total genesis supply of Ada: 11000000000.000000
[db-sync-node:Info:4] [2020-06-06 11:25:32.80 UTC] epochPluginOnStartup: Checking
[db-sync-node:Info:4] [2020-06-06 11:25:32.81 UTC] localInitiatorNetworkApplication: connecting to node via "/run/cardano-node/node.socket"
[db-sync-node.Handshake:Info:20] [2020-06-06 11:25:32.81 UTC] [String "Send MsgProposeVersions (fromList [(NodeToClientV_2,TInt 42)])",String "LocalHandshakeTrace",String "ConnectionId {localAddress = LocalAddress {getFilePath = \"\"}, r>
[db-sync-node.Handshake:Info:20] [2020-06-06 11:25:32.81 UTC] [String "Recv MsgAcceptVersion NodeToClientV_2 (TInt 42)",String "LocalHandshakeTrace",String "ConnectionId {localAddress = LocalAddress {getFilePath = \"\"}, remoteAddress = >
[db-sync-node:Info:25] [2020-06-06 11:25:32.81 UTC] Starting chainSyncClient
[db-sync-node:Info:25] [2020-06-06 11:25:32.83 UTC] Cardano.Db tip is at empty (genesis)
[db-sync-node:Info:29] [2020-06-06 11:25:32.83 UTC] Running DB thread
[db-sync-node:Info:29] [2020-06-06 11:25:33.20 UTC] insertShelleyBlock: slot 0, block 0, hash 48b458d41a00acad75b02658e2de3cd4e87f7b3804aaab56aa9d418ce44c2cb7
[db-sync-node:Info:29] [2020-06-06 11:25:33.92 UTC] epochPluginInsertBlock: Inserting row in epoch table for epoch 0
[db-sync-node:Info:29] [2020-06-06 11:25:34.03 UTC] epochPluginInsertBlock: Inserting row in epoch table for epoch 1
[db-sync-node:Info:29] [2020-06-06 11:25:34.11 UTC] epochPluginInsertBlock: Inserting row in epoch table for epoch 2
[db-sync-node:Info:29] [2020-06-06 11:25:34.19 UTC] epochPluginInsertBlock: Inserting row in epoch table for epoch 3
[db-sync-node:Info:29] [2020-06-06 11:25:34.28 UTC] epochPluginInsertBlock: Inserting row in epoch table for epoch 4
[db-sync-node:Info:29] [2020-06-06 11:25:34.36 UTC] epochPluginInsertBlock: Inserting row in epoch table for epoch 5
[db-sync-node:Info:29] [2020-06-06 11:25:34.44 UTC] epochPluginInsertBlock: Inserting row in epoch table for epoch 6
[db-sync-node:Error:29] [2020-06-06 11:25:34.46 UTC] DB lookup fail in insertTxIn: tx hash 46dc1395de27ce13f943d509ce701b61d25a6828bb169b72521d47cf41fc591a
[db-sync-node:Info:29] [2020-06-06 11:25:34.46 UTC] Shutting down DB thread
[db-sync-node:Error:32] [2020-06-06 11:25:34.46 UTC] recvMsgRollForward: AsyncCancelled

Reproduced locally.

The whole things is still very much a work in progress. I only just got this working on Thursday and it has only have minimal testing. This particular error is due the the representation of addresses in the genesis distribution changing in one of the underlying libraries.

I know of at least 1 more breaking change that is going to happen in the underlying libraries.

Spent the last 2+ hours investigatong this. The DB lookup fail in insertTxIn is due caused by the failure to find the UTxO output referenced in an on chain transaction. This in term is highly likely due the fact that the consensus layer and db-sync both have code that takes Shelley genesis and converts it into an initial UTxO set. If the two implementations of thjs code are not operationally identical, we will hit this failure.

The solution is to have one bit of code to convert the Shelley Genesis to a UTxO set and that code should probably live in cardano-ledger-specs .

Started trying to move the ShelleyGenesis related stuff to cardano-ledger-specs and ran into some rather hairy problems. This is very unlikely to be solved today or even tomorrow.

https://github.com/input-output-hk/cardano-ledger-specs/issues/1529
https://github.com/input-output-hk/cardano-ledger-specs/issues/1530

any ETA guys? @erikd @rhyslbw thx

How close to the bleeding edge do you want to be? I have a WIP branch that currently seems to be working correctly on F&F, but its raw as all hell and not even close to being properly tested.

Its 9pm here. The branch is erikd/shelley-32byte-address-hash. If you break it you get to keep both pieces!

thank you checked > works > closing.

reopening, since the fix is not merged to master yet.

This has been merged to master. Closing.