General-grants-program: Create a sentry node configuration for Polkadot validators with guide

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

__This issue now has a funding of 150.0 DAI (150.0 USD @ $1.0/DAI) attached to it.__

• If you would like to work on this issue you can 'start work' on the Gitcoin Issue Details page.
  
  
  
  • Want to chip in? Add your own contribution here.
  
  
  • Questions? Checkout Gitcoin Help or the Gitcoin Slack
  
  
  • $80,240.14 more funded OSS Work available on the Gitcoin Issue Explorer
  
  

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

__Workers have applied to start work__.

These users each claimed they can complete the work by 12 months from now.
Please review their action plans below:

1) gutsal-arsen has applied to start work _(Funders only: approve worker | reject worker)_.

Create 2-nodes configuration (sentry and validator), create a deployment guide.

Learn more on the Gitcoin Issue Details page.

@Web3Foundation applied on Gitcoin. Could you approve?

Hey @agutsal gitcoin is down at the moment once services resume will review!

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

__Work has been started__.

These users each claimed they can complete the work by 9 months, 1 week from now.
Please review their action plans below:

1) gutsal-arsen has been approved to start work.

Create 2-nodes configuration (sentry and validator), create a deployment guide.

Learn more on the Gitcoin Issue Details page.

@agutsal you've been approved to start work.

@gutsal-arsen Hello from Gitcoin Core - are you still working on this issue? Please submit a WIP PR or comment back within the next 3 days or you will be removed from this ticket and it will be returned to an ‘Open’ status. Please let us know if you have questions!

• [x] reminder (3 days)
• [ ] escalation to mods (6 days)

Funders only: Snooze warnings for 1 day | 3 days | 5 days | 10 days | 100 days

@gutsal-arsen Hello from Gitcoin Core - are you still working on this issue? Please submit a WIP PR or comment back within the next 3 days or you will be removed from this ticket and it will be returned to an ‘Open’ status. Please let us know if you have questions!

• [x] reminder (3 days)
• [ ] escalation to mods (6 days)

Funders only: Snooze warnings for 1 day | 3 days | 5 days | 10 days | 100 days

@gutsal-arsen Hello from Gitcoin Core - are you still working on this issue? Please submit a WIP PR or comment back within the next 3 days or you will be removed from this ticket and it will be returned to an ‘Open’ status. Please let us know if you have questions!

• [x] reminder (3 days)
• [ ] escalation to mods (6 days)

Funders only: Snooze warnings for 1 day | 3 days | 5 days | 10 days | 100 days

@gutsal-arsen Hello from Gitcoin Core - are you still working on this issue? Please submit a WIP PR or comment back within the next 3 days or you will be removed from this ticket and it will be returned to an ‘Open’ status. Please let us know if you have questions!

• [x] reminder (3 days)
• [ ] escalation to mods (6 days)

Funders only: Snooze warnings for 1 day | 3 days | 5 days | 10 days | 100 days

@gutsal-arsen Hello from Gitcoin Core - are you still working on this issue? Please submit a WIP PR or comment back within the next 3 days or you will be removed from this ticket and it will be returned to an ‘Open’ status. Please let us know if you have questions!

• [x] reminder (3 days)
• [ ] escalation to mods (6 days)

Funders only: Snooze warnings for 1 day | 3 days | 5 days | 10 days | 100 days

@gitcoinbot alive, will update soon

@gutsal-arsen Hello from Gitcoin Core - are you still working on this issue? Please submit a WIP PR or comment back within the next 3 days or you will be removed from this ticket and it will be returned to an ‘Open’ status. Please let us know if you have questions!

• [x] reminder (3 days)
• [ ] escalation to mods (6 days)

Funders only: Snooze warnings for 1 day | 3 days | 5 days | 10 days | 100 days

@agutsal great look forward to it! :)

@gutsal-arsen Hello from Gitcoin Core - are you still working on this issue? Please submit a WIP PR or comment back within the next 3 days or you will be removed from this ticket and it will be returned to an ‘Open’ status. Please let us know if you have questions!

• [x] reminder (3 days)
• [ ] escalation to mods (6 days)

Funders only: Snooze warnings for 1 day | 3 days | 5 days | 10 days | 100 days

@gitcoinbot wait abit

@gutsal-arsen Hello from Gitcoin Core - are you still working on this issue? Please submit a WIP PR or comment back within the next 3 days or you will be removed from this ticket and it will be returned to an ‘Open’ status. Please let us know if you have questions!

• [x] reminder (3 days)
• [ ] escalation to mods (6 days)

Funders only: Snooze warnings for 1 day | 3 days | 5 days | 10 days | 100 days

@gitcoinbot working

@gutsal-arsen Hello from Gitcoin Core - are you still working on this issue? Please submit a WIP PR or comment back within the next 3 days or you will be removed from this ticket and it will be returned to an ‘Open’ status. Please let us know if you have questions!

• [x] reminder (3 days)
• [ ] escalation to mods (6 days)

Funders only: Snooze warnings for 1 day | 3 days | 5 days | 10 days | 100 days

@gutsal-arsen Hello from Gitcoin Core - are you still working on this issue? Please submit a WIP PR or comment back within the next 3 days or you will be removed from this ticket and it will be returned to an ‘Open’ status. Please let us know if you have questions!

• [x] reminder (3 days)
• [ ] escalation to mods (6 days)

Funders only: Snooze warnings for 1 day | 3 days | 5 days | 10 days | 100 days

@Web3Foundation tried to build using polkadot Dockerfile.
docker build --tag polkadot -f Dockerfile . command fails with:

Tried twice - same result.

@agutsal Are you building from the master branch or v0.4 branch? It might be better to build from v0.4 since its more stable and will connect to Alexander testnet.

Please reach out to @logan:web3.foundation on Riot for more immediate communication.

Even worse on v0.4:

@lsaether @Web3Foundation

@lsaether what is Riot? Gimme the link plz

This should work

This looks like you are not compiling the Wasm binary before running cargo build, try to run the script in scripts/build.sh first

@lsaether I'm just running docker/build.sh yet. I assume it works, but from what I see it does not ;)

@gutsal-arsen Hello from Gitcoin Core - are you still working on this issue? Please submit a WIP PR or comment back within the next 3 days or you will be removed from this ticket and it will be returned to an ‘Open’ status. Please let us know if you have questions!

• [x] reminder (3 days)
• [ ] escalation to mods (6 days)

Funders only: Snooze warnings for 1 day | 3 days | 5 days | 10 days | 100 days

@lsaether @Web3Foundation seems I've fixed Dockerfile PR #308

Please merge.

@gutsal-arsen Hello from Gitcoin Core - are you still working on this issue? Please submit a WIP PR or comment back within the next 3 days or you will be removed from this ticket and it will be returned to an ‘Open’ status. Please let us know if you have questions!

• [x] reminder (3 days)
• [ ] escalation to mods (6 days)

Funders only: Snooze warnings for 1 day | 3 days | 5 days | 10 days | 100 days

@gitcoinbot in progress

@gutsal-arsen Hello from Gitcoin Core - are you still working on this issue? Please submit a WIP PR or comment back within the next 3 days or you will be removed from this ticket and it will be returned to an ‘Open’ status. Please let us know if you have questions!

• [x] reminder (3 days)
• [ ] escalation to mods (6 days)

Funders only: Snooze warnings for 1 day | 3 days | 5 days | 10 days | 100 days

@gutsal-arsen Hello from Gitcoin Core - are you still working on this issue? Please submit a WIP PR or comment back within the next 3 days or you will be removed from this ticket and it will be returned to an ‘Open’ status. Please let us know if you have questions!

• [x] reminder (3 days)
• [ ] escalation to mods (6 days)

Funders only: Snooze warnings for 1 day | 3 days | 5 days | 10 days | 100 days

@gutsal-arsen Hello from Gitcoin Core - are you still working on this issue? Please submit a WIP PR or comment back within the next 3 days or you will be removed from this ticket and it will be returned to an ‘Open’ status. Please let us know if you have questions!

• [x] reminder (3 days)
• [ ] escalation to mods (6 days)

Funders only: Snooze warnings for 1 day | 3 days | 5 days | 10 days | 100 days

@gitcoinbot speadking on PR #308

@gutsal-arsen Hello from Gitcoin Core - are you still working on this issue? Please submit a WIP PR or comment back within the next 3 days or you will be removed from this ticket and it will be returned to an ‘Open’ status. Please let us know if you have questions!

• [x] reminder (3 days)
• [ ] escalation to mods (6 days)

Funders only: Snooze warnings for 1 day | 3 days | 5 days | 10 days | 100 days

@gitcoinbot shut up please

@gutsal-arsen Hello from Gitcoin Core - are you still working on this issue? Please submit a WIP PR or comment back within the next 3 days or you will be removed from this ticket and it will be returned to an ‘Open’ status. Please let us know if you have questions!

• [x] reminder (3 days)
• [ ] escalation to mods (6 days)

Funders only: Snooze warnings for 1 day | 3 days | 5 days | 10 days | 100 days

@gitcoinbot

@gutsal-arsen Hello from Gitcoin Core - are you still working on this issue? Please submit a WIP PR or comment back within the next 3 days or you will be removed from this ticket and it will be returned to an ‘Open’ status. Please let us know if you have questions!

• [x] reminder (3 days)
• [ ] escalation to mods (6 days)

Funders only: Snooze warnings for 1 day | 3 days | 5 days | 10 days | 100 days

@gutsal-arsen Hello from Gitcoin Core - are you still working on this issue? Please submit a WIP PR or comment back within the next 3 days or you will be removed from this ticket and it will be returned to an ‘Open’ status. Please let us know if you have questions!

• [x] reminder (3 days)
• [ ] escalation to mods (6 days)

Funders only: Snooze warnings for 1 day | 3 days | 5 days | 10 days | 100 days

@gutsal-arsen Hello from Gitcoin Core - are you still working on this issue? Please submit a WIP PR or comment back within the next 3 days or you will be removed from this ticket and it will be returned to an ‘Open’ status. Please let us know if you have questions!

• [x] reminder (3 days)
• [ ] escalation to mods (6 days)

Funders only: Snooze warnings for 1 day | 3 days | 5 days | 10 days | 100 days

@gitcoinbot just fnished discussion on recent WIP PR continue to work

@gutsal-arsen Hello from Gitcoin Core - are you still working on this issue? Please submit a WIP PR or comment back within the next 3 days or you will be removed from this ticket and it will be returned to an ‘Open’ status. Please let us know if you have questions!

• [x] reminder (3 days)
• [ ] escalation to mods (6 days)

Funders only: Snooze warnings for 1 day | 3 days | 5 days | 10 days | 100 days

@gutsal-arsen Hello from Gitcoin Core - are you still working on this issue? Please submit a WIP PR or comment back within the next 3 days or you will be removed from this ticket and it will be returned to an ‘Open’ status. Please let us know if you have questions!

• [x] reminder (3 days)
• [ ] escalation to mods (6 days)

Funders only: Snooze warnings for 1 day | 3 days | 5 days | 10 days | 100 days

lol sorry i've muted @gitcoinbot for 100 days @agutsal

@gutsal-arsen Hello from Gitcoin Core - are you still working on this issue? Please submit a WIP PR or comment back within the next 3 days or you will be removed from this ticket and it will be returned to an ‘Open’ status. Please let us know if you have questions!

• [x] reminder (3 days)
• [ ] escalation to mods (6 days)

Funders only: Snooze warnings for 1 day | 3 days | 5 days | 10 days | 100 days

@Web3Foundation thx

@gutsal-arsen Hello from Gitcoin Core - are you still working on this issue? Please submit a WIP PR or comment back within the next 3 days or you will be removed from this ticket and it will be returned to an ‘Open’ status. Please let us know if you have questions!

• [x] reminder (3 days)
• [ ] escalation to mods (6 days)

Funders only: Snooze warnings for 1 day | 3 days | 5 days | 10 days | 100 days

@Web3Foundation seems mute does not work ;(
@gitcoinbot yes, I'm still working on docker-compose network configuration to run one node behind another. @Web3Foundation plz, confirm that's what you expect.

@gutsal-arsen Hello from Gitcoin Core - are you still working on this issue? Please submit a WIP PR or comment back within the next 3 days or you will be removed from this ticket and it will be returned to an ‘Open’ status. Please let us know if you have questions!

• [x] reminder (3 days)
• [ ] escalation to mods (6 days)

Funders only: Snooze warnings for 1 day | 3 days | 5 days | 10 days | 100 days

@gutsal-arsen Hello from Gitcoin Core - are you still working on this issue? Please submit a WIP PR or comment back within the next 3 days or you will be removed from this ticket and it will be returned to an ‘Open’ status. Please let us know if you have questions!

• [x] reminder (3 days)
• [ ] escalation to mods (6 days)

Funders only: Snooze warnings for 1 day | 3 days | 5 days | 10 days | 100 days

@gutsal-arsen Hello from Gitcoin Core - are you still working on this issue? Please submit a WIP PR or comment back within the next 3 days or you will be removed from this ticket and it will be returned to an ‘Open’ status. Please let us know if you have questions!

• [x] reminder (3 days)
• [ ] escalation to mods (6 days)

Funders only: Snooze warnings for 1 day | 3 days | 5 days | 10 days | 100 days

@Web3Foundation I'm still waiting for your response on my explanation in previous comment above. Is docker-compose custom virtual network configuration what you expect to have 2 nodes running one behind another?

Hey @agutsal sorry for the long response; @lsaether or @fgimenez could probably chime in here to the question about

"is docker-compose custom virtual network configuration what you expect to have 2 nodes running one behind another?"

Probably @lsaether @fgimenez are both on vacation, @Web3Foundation ? ;)

Will ping again now, @agutsal things have just been super busy, apologies again for the wait.

Correct, two nodes, one running "behind" the other. The sentry node is facing the public network (and connects to the rest of the p2p network). The validator node is behind a firewall and can only communicate with the sentry node.

* public network * <---> Sentry node <-- firewall --> Validator node

https://guide.kusama.network/en/latest/try/secure-validator-setup/

Feel free to ping me if you have other questions.

@laboon once again - we're both speaking about Compose private network configuration.

Apologies, I just got pointed to this and misunderstood your question. Federico, who I believe put this request together, is on vacation this week.

_"is docker-compose custom virtual network configuration what you expect to have 2 nodes running one behind another?""_

Using Docker Compose to make an isolated private network is certainly one way to do it and should be acceptable. I don't know if that was the original expectation, however (I don't see anything to the contrary here - not sure if you have private communications elsewhere).

Feel free to let me know if I didn't answer your question.

@agutsal as I see it the overlay networks created by docker-compose are not enough to isolate a validator node from the rest of the polkadot network; as soon as the validator connects to its boot nodes it will be known by the network, and if you don't put in place any additional measure other peer nodes will be able to connect to it. You can verify all this by starting the network with a docker compose file like this:

version: '3'
services:
  node:
    image: parity/polkadot:v0.4.4
    container_name: polkadot-node 
    command: >
      --chain=alexander --validator 
    networks: 
      testing_net: 
        ipv4_address: 172.28.1.1 

networks:
  testing_net:
    ipam:        
      driver: default 
      config:            
        - subnet: 172.28.0.0/16  

This single node has an internal ip address, doesn't declare any p2p port to be accessed from outside and doesn't have any additional peer in the local overlay network. When i start the node:

$ docker-compose -f dcp.yml up --force-recreate 
Recreating polkadot-node ... done
Attaching to polkadot-node
polkadot-node | 2019-08-12 08:52:16 Parity Polkadot
polkadot-node | 2019-08-12 08:52:16   version 0.4.4-aa49754-x86_64-linux-gnu
polkadot-node | 2019-08-12 08:52:16   by Parity Team <[email protected]>, 2017-2019
polkadot-node | 2019-08-12 08:52:16 Chain specification: Alexander
polkadot-node | 2019-08-12 08:52:16 Node name: dcp-test
polkadot-node | 2019-08-12 08:52:16 Roles: AUTHORITY
polkadot-node | 2019-08-12 08:52:16 Generated a new keypair: 568dbb11bcc8f4bd1480d381e2ee87f9e1ff3798c99efa23f550dcce7417a506 (5E2C5Usq...)
polkadot-node | 2019-08-12 08:52:16 Initializing Genesis block/state (state: 0xb7d6…2707, header-hash: 0xdcd1…025b)
polkadot-node | 2019-08-12 08:52:16 Loaded block-time = 6 seconds from genesis on first-launch
polkadot-node | 2019-08-12 08:52:16 Loading GRANDPA authority set from genesis on what appears to be first startup.
polkadot-node | 2019-08-12 08:52:16 Best block: #0
polkadot-node | 2019-08-12 08:52:16 Local node identity is: QmWg2c4V6qq1hFYUcbkmckUgtMfWiC7aq5u9e8X5Sq45Tu
polkadot-node | 2019-08-12 08:52:16 Listening for new connections on 127.0.0.1:9944.
polkadot-node | 2019-08-12 08:52:17 Using authority key 5E2C5Usqi8UhQLCB1CGobhB5aGMg6zWxvg2h7Vq39tWbj14Q
polkadot-node | 2019-08-12 08:52:18 Discovered external node address: /ip4/92.176.206.107/tcp/30333/p2p/QmWg2c4V6qq1hFYUcbkmckUgtMfWiC7aq5u9e8X5Sq45Tu
polkadot-node | 2019-08-12 08:52:22 Syncing, target=#2614309 (2 peers), best: #256 (0x7e81…c0f7), finalized #0 (0xdcd1…025b), ⬇ 84.0kiB/s ⬆ 5.9kiB/s
polkadot-node | 2019-08-12 08:52:28 Syncing 93.9 bps, target=#2614310 (4 peers), best: #896 (0x6da8…cd8f), finalized #0 (0xdcd1…025b), ⬇ 30.6kiB/s ⬆ 1.7kiB/s
polkadot-node | 2019-08-12 08:52:35 Syncing 94.4 bps, target=#2614311 (5 peers), best: #1536 (0xf9b0…8351), finalized #0 (0xdcd1…025b), ⬇ 34.8kiB/s ⬆ 3.1kiB/s
polkadot-node | 2019-08-12 08:52:41 Syncing 93.9 bps, target=#2614312 (5 peers), best: #2048 (0xaa1c…3b27), finalized #0 (0xdcd1…025b), ⬇ 40.4kiB/s ⬆ 2.8kiB/s
polkadot-node | 2019-08-12 08:52:46 Syncing 93.4 bps, target=#2614313 (5 peers), best: #2515 (0x8bff…4f0b), finalized #0 (0xdcd1…025b), ⬇ 7.6kiB/s ⬆ 1.3kiB/s
polkadot-node | 2019-08-12 08:52:51 Syncing 93.2 bps, target=#2614314 (5 peers), best: #2981 (0x47ab…67bd), finalized #0 (0xdcd1…025b), ⬇ 6.0kiB/s ⬆ 0.3kiB/s

[..........]

You can see that other peers can connect to it easily right after start.

However, i think the overlay network is a good step in the right direction, it creates a network address space only known by the members of the network, and you could configure the validator and the public nodes to restrict access, maybe using the polkadot binary options listen-addr and reserved-nodes. We proposed a secure validator setup that uses this approach but, instead of using a docker overlay network for the private network address space, it uses a VPN for increased security. Take a look here https://hackmd.io/QSJlqjZpQBihEU_ojmtR8g#Conclusions-and-Proposal

@fgimenez as mentioned here wouldn't this work:

# Docker compose file to simulate a sentry node setup.
#
#
# Setup:
#
# Validator A is not supposed to be connected to the public internet. Instead it
# connects to a sentry node (sentry-a) which connects to the public internet.
# Validator B can reach validator A via sentry node A and vice versa.
#
#
# Usage:
#
# 1. Build `target/release/substrate` binary: `cargo build --release`
#
# 2. Start networks and containers: `sudo docker-compose -f scripts/sentry-node/docker-compose.yml up`
#
# 3. Reach:
#   - polkadot/apps on localhost:3000
#   - validator-a: localhost:9944
#   - validator-b: localhost:9945
#   - sentry-a: localhost:9946

version: "3.7"
services:

  validator-a:
    ports:
      - "9944:9944"
    volumes:
      - ../../target/release/substrate:/usr/local/bin/substrate
    image: parity/substrate
    networks:
      - network-a
    command:
      # Local node id: QmRpheLN4JWdAnY7HGJfWFNbfkQCb6tFf4vvA6hgjMZKrR
      - "--node-key"
      - "0000000000000000000000000000000000000000000000000000000000000001"
      - "--base-path"
      - "/tmp/alice"
      - "--chain=local"
      - "--key"
      - "//Alice"
      - "--port"
      - "30333"
      - "--validator"
      - "--name"
      - "AlicesNode"
      - "--reserved-nodes"
      - "/dns4/sentry-a/tcp/30333/p2p/QmV7EhW6J6KgmNdr558RH1mPx2xGGznW7At4BhXzntRFsi"
      # Not only bind to localhost.
      - "--ws-external"
      - "--rpc-external"
      # - "--log"
      # - "sub-libp2p=trace"
      # - "--log"
      # - "afg=trace"
      - "--no-telemetry"
      - "--rpc-cors"
      - "all"

  sentry-a:
    image: parity/substrate
    ports:
      - "9946:9944"
    volumes:
      - ../../target/release/substrate:/usr/local/bin/substrate
    networks:
      - network-a
      - internet
    command:
      # Local node id: QmV7EhW6J6KgmNdr558RH1mPx2xGGznW7At4BhXzntRFsi
      - "--node-key"
      - "0000000000000000000000000000000000000000000000000000000000000003"
      - "--base-path"
      - "/tmp/sentry"
      - "--chain=local"
      # Don't configure a key, as sentry-a is not a validator.
      # - "--key"
      # - "//Charlie"
      - "--port"
      - "30333"
      # sentry-a is not a validator.
      # - "--validator"
      - "--name"
      - "CharliesNode"
      - "--bootnodes"
      - "/dns4/validator-a/tcp/30333/p2p/QmRpheLN4JWdAnY7HGJfWFNbfkQCb6tFf4vvA6hgjMZKrR"
      - "--bootnodes"
      - "/dns4/validator-b/tcp/30333/p2p/QmSVnNf9HwVMT1Y4cK1P6aoJcEZjmoTXpjKBmAABLMnZEk"
      - "--no-telemetry"
      - "--rpc-cors"
      - "all"
      # Not only bind to localhost.
      - "--ws-external"
      - "--rpc-external"
      # Make sure sentry-a still participates as a grandpa voter to forward
      # grandpa finality gossip messages.
      - "--grandpa-voter"

  validator-b:
    image: parity/substrate
    ports:
      - "9945:9944"
    volumes:
      - ../../target/release/substrate:/usr/local/bin/substrate
    networks:
      - internet
    command:
      # Local node id: QmSVnNf9HwVMT1Y4cK1P6aoJcEZjmoTXpjKBmAABLMnZEk
      - "--node-key"
      - "0000000000000000000000000000000000000000000000000000000000000002"
      - "--base-path"
      - "/tmp/bob"
      - "--chain=local"
      - "--key"
      - "//Bob"
      - "--port"
      - "30333"
      - "--validator"
      - "--name"
      - "BobsNode"
      - "--bootnodes"
      - "/dns4/validator-a/tcp/30333/p2p/QmRpheLN4JWdAnY7HGJfWFNbfkQCb6tFf4vvA6hgjMZKrR"
      - "--bootnodes"
      - "/dns4/sentry-a/tcp/30333/p2p/QmV7EhW6J6KgmNdr558RH1mPx2xGGznW7At4BhXzntRFsi"
      - "--no-telemetry"
      - "--rpc-cors"
      - "all"
      # Not only bind to localhost.
      - "--ws-external"
      - "--rpc-external"

  ui:
    image: polkadot-js/apps
    ports:
      - "3000:80"

networks:
  network-a:
  internet:

@fgimenez well, I'd consider that approach after few days of silence

@agutsal sure sorry for the late response, combining reserved-nodes and bootnodes is a good option. As stated in my previous comment I would also introduce listen-addr in the validator with the validator ip in the docker compose overlay network, so that you make sure that the validator only accepts connections from peers in that address space.

Thanks for your reply. From what I see they use substrate docker image which we are not.

Basically should I follow this:
docker run -ti parity/polkadot:v0.4.4 --chain=alex --alice <== run sentry
it produce something like this where I should check for Discovered external node address

then pass it to validator like that:
docker run -ti parity/polkadot:v0.4.4 --chain=local --validator --reserved-nodes /ip4/5.58.235.221/tcp/30333/p2p/QmVUPpkU2LoBMqhagLmF6H2Xnb4ShK8KV2ZzKeJkmGGBkX

If that's correct where to pass bootnodes? If not - fix me plz, @fgimenez

As i see it you should not use any externel discovered address, instead if you should use a multiaddr that includes the ip of the public node on the docker compose overlay network. Not sure if you can know it before the docker compose network is launched.

I only mentioned bootnodes because they appeared in the example you l8nked, you only need them if you are going to create a local network. If you are going to join alex or (what would be better imo) kusama, the bootnodes are already defined in the chainspec.

@fgimenez as per requirenment validator node should connect to sentry node. I need to know how to do that. That's it.

Yes, this is done by setting --reserved-nodes in the validator with multiaddr of the public nodes (one reserved-nodes per public node). It should be better to use here the private address of the public node for the multiaddr, does that make sense?

@fgimenez ;) could be, just don't know the format of --reserved-nodes paramenter specifying the multiaddr. Would be thankful for sample.

sure, something like this /ip4/<private_ip>/tcp/30333/p2p/<peer_id>

You should set the private ip as the one assigned to the public node in the docker compose overlay network. The peer id is randomly assigned on boot if you don't specify a node key with --node-key. This node key should be a 64 hex char string, you can try starting a node locally by setting a known node-key and then checking which peer id is assigned, and use that peer id to construct the reserved node multiaddr.

So, for each public node:

• start a local node with polkadot --node-key=<your_node_key>
• take note of the assigned peer id
• set the public node start command to include --node-key to the previous value, so that you make sure that it will always have the same peer id
• construct the multiaddr as /ip4/<private_ip>/tcp/30333/p2p/<peer_id>

and start the validator with one --reserved-nodes set to each public node multiaddr

Let me know if you need anything else.

@fgimenez I"m currently thinking how to make this 2 step process in a single docker-compose.yml file. Tell me - <your_node_key> could be randomly generated as some UUID?

Something like that: $(openssl rand -base64 500 | tr -dc 'a-zA-Z0-9' | fold -w 45 | head -n 1)

You can use one of the libp2p libraries for generating the peer id and associated keypair, for instance https://github.com/libp2p/js-peer-id node-key is the private key, with 256 bits of length.

@fgimenez plz, review my WIP PR

from within docker directory just run docker-compose -f sentry-docker-compose.yml up and see the output. Seems validator succesfuly connects to sentry as a peer.

If accepted, I'll add some more configuration and push to be merged into master.

Sure will take a look next week :+1:

@fgimenez some update been pushed to my WIP RP
Please, let me know after review.

@agutsal looks good, it's pretty similar to https://github.com/paritytech/substrate/blob/master/scripts/sentry-node/docker-compose.yml, right? What's the benefit of adding it to polkadot's repo?

Well @fgimenez, it's actually based on sentry-node/docker-compose.yml but has significant differences: we can pass bunch of environment variables to control both containers:

{SENTRY,VALIDATOR}_NODE_KEY
{SENTRY,VALIDATOR}_BASE_PATH
{SENTRY,VALIDATOR}_CHAIN
{SENTRY,VALIDATOR}_PORT
{SENTRY,VALIDATOR}_NAME
VALIDATOR_RESERVED_NODES
SENTRY_BOOTNODES

This is a must to run sentry/validator with unique NODE_KEY, RESERVED_NODES/BOOTNODES URI on public (not local) chain.
By default it runs with 0000000000000000000000000000000000000000000000000000000000000001
0000000000000000000000000000000000000000000000000000000000000003

which is OK for testing but not for production. Also some redundant parameters been cleared.

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

__Work for 150.0 DAI (150.0 USD @ $1.0/DAI) has been submitted by__:

1. @gutsal-arsen

@Web3Foundation please take a look at the submitted work:

• PR by @gutsal-arsen

• Learn more on the Gitcoin Issue Details page
• Want to chip in? Add your own contribution here.
• Questions? Checkout Gitcoin Help or the Gitcoin Slack
• $138,268.93 more funded OSS Work available on the Gitcoin Issue Explorer

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

__The funding of 150.0 DAI (150.0 USD @ $1.0/DAI) attached to this issue has been approved & issued to @gutsal-arsen.__

• Learn more on the Gitcoin Issue Details page
• Questions? Checkout Gitcoin Help or the Gitcoin Slack
• $139,333.62 more funded OSS Work available on the Gitcoin Issue Explorer

@Web3Foundation Thanks for your payment. However, I think I also have to fix docs in scope of current project. Current repository does not contain docs, should I clone https://github.com/w3f/polkadot-wiki and fix them there?

@agutsal sure; there was a bit of a repeated effort as @fgimenez had created a tool that maybe more successfully solves this problem. We paid out because of your repeated efforts in the ecosystem and the ongoing respective time you've taken in past bounties & this one.

Feel free to amend documentation and leave any comments for review.

Thanks for letting me know. Would work on that today and let you know @fgimenez @Web3Foundation.

Closing since sentry nodes have been deprecated.