General-grants-program: Open Source Block Explorer

I started working on the Polkadot (or generalized Substrate) Explorer today.

Short intro: my name is Emiel and I am organizer of the Polkadot NL meetup (inaugural meetup on 2 Oct 2018 in Rotterdam). Furthermore I am working on a multi-chain block explorer called WEB3SCAN. An early prototype for this multi-chain explorer is available at https://explorer.web3scan.net

THE PLAN
I intend to build a Block Explorer for Polkadot (or more generalized for any Substrate Instantiation). The idea is to start with a stack I have used for building WEB3SCAN's 'Multi-chain EVM Block Explorer'.
The stack consists of:

1) a harvester (which fetches RPC-calls and puts it in a DB, indexes, enriches, aggregates the data)
2) a RDBMS (with all the indexed, enriched and aggregated data)
3) an API-layer (with fat API methods for the various objects in the RDBMS)
4) a GUI (a user interface to the API layer / RDBMS)

Although this project will initially be developed under its own project name (POLKASCAN) it will eventually likely move under the WEB3SCAN umbrella (and the generalized multi-chain explorer mentioned above).

I will mostly be working on this alone (full-time) for the upcoming two months. The short term goals is to prepare for a presentation called: 'Building a Polkadot/Substrate Explorer' for the inaugural Polkadot NL Meetup in Rotterdam on 2 October 2018.

WHAT WE NEED
We have most of the experience in house for buidling Block Explorers, however we do very much need the assistance of Web3 Foundation to mobilize Parity to add some very much needed extra endpoints / methods to the Polkadot Client. These requirements will mature in the upcoming weeks as we start with our initial development.

Example: One clear and early example is we are currently able to harvest the block headers from chaintip all the way back to genesis, but there is currently no way (yet) to get data on the block body such as 'extrinsics' (transactions). Of course I understand that the tech is in a very early stage but I hope to contribute to make the client as friendly as possible for Block Explorer style ecosystem projects.

Sounds great, looking forward to updates.
I heard that @tomusdrw is working on an RPC to get extrinsics for a block by hash already!

Many intermediate updates have been made on Twitter:

• Style/Branding: https://twitter.com/polkascan/status/1037706833241227265
• (G)UI: https://twitter.com/polkascan/status/1037715325146550278

• Daily Quiz on the data:

• Q8: https://twitter.com/polkascan/status/1037676638413250560

• Q7: https://twitter.com/polkascan/status/1037321435415568384
• Q6: https://twitter.com/polkascan/status/1036920562461880320
• Q5: https://twitter.com/polkascan/status/1036585024185876480
• Q4: https://twitter.com/polkascan/status/1036204125862674432
• Q3: https://twitter.com/polkascan/status/1036200731144007680
• Q2: https://twitter.com/polkascan/status/1035465166450831360
• Q1: https://twitter.com/polkascan/status/1035089580268109824

More updates soon.

Twitter thread with screen designs for the Polkadot Explorer 'Polkascan': https://twitter.com/polkascan/status/1039501738842226689

A first (static but browsable) version of the Polkascan Explorer is available at https://polkascan.io Go check it out and let us know what you think!!

https://twitter.com/polkascan/status/1042745927600099328

Great start/progress @emielvanderhoek

Update:

Polkascan (https://polkascan.io) now supports multiple chains. We currently show allow exploration of the following chains:

1. Polkadot (poc-2): https://polkascan.io/n-poc2/poc-2/ (This is the relay chain)
2. BBQ-Birch (master) Parachain Candidate: https://polkascan.io/n-pre/bbqbirch/ (This is a WASM smart contract platform)

Please note that these links may break at any time due to upgrades. Follow the the links from the main entry at https://polkascan.io for current click paths.

We chose for a design with a colour-code per network. Currently the Polkadot Relaychain has a pink colour and the BBQ Birch chain has a green colour.

We are NOT updating blocks in realtime atm. Currently we work with data snapshots. Real-time block updates of the networks supported by Polkascan will follow some time after the Polkadot Relaychain moves to POC-3.

With this update Polkscan is capable of supporting ANY* Substrate instance. In our current version we stripped everything to the bare minimum which is identical for any Substrate instance. Current ‘master’ and the next POC3 for Polkadot will (if I am correct) only differ through the Runtime. These are all objects that differentiate one Substrate instance from another.

The runtime objects (calls, events and storage function) are specified through the metadata which you can get (and decode) through RPC: chain_getMetadata(). This Metadata should be seen as the ABI for te runtime. Polkascan will show the (decoded) runtime metadata object in the UI later this week.

We are currently working out how to support those flexible runtime objects in the Polkascan explorer. E.g. the Polkadot Relaychain runtime has a ‘parachains’ object and the BBQ Birch chain does not.

Twitter updates (and some images):
https://twitter.com/polkascan/status/1051837686166814721
https://twitter.com/polkadotnetwork/status/1052036003219697664
https://twitter.com/emielvanderhoek/status/1051841796089761792

Update:

Polkascan (BBQ Birch) now contains all runtime events.
BBQ Birch Events: https://polkascan.io/n-pre/bbqbirch/activity/event/
BBQ Birch Events (extrinsic triggered): https://polkascan.io/n-pre/bbqbirch/activity/event/extrinsic/
BBQ Birch Events (other): https://polkascan.io/n-pre/bbqbirch/activity/event/finalization/
The event detail pages show the decoded event parameters.

Next week I’ll add filtered event views such as:
1) balance.transfers
2) staking.rewards
3) some of the treasury events.

Polkascan (Polkadot) will have these features when POC-3 arrives (soon).
Also: the extrinsic detail pages now also contain the decoded extrinsic parameters.