Celo blockchain client next steps (what does the community want?)

Hello, validators, node maintainers dapp builders and users!

I’m a member of the blockchain team at clabs and I’m reaching out to the
community to help build our understanding of what people really want from the
celo blockchain client.

We’re interested to understand from the community’s perspective what is good
about the celo-blockchain-client and also what is bad, what causes you
problems, are there areas where it falls short of your expectations?

The blockchain team’s current focus is on scalability and we are looking to
integrate a new mechanism for block construction that should provide
significant performance improvements (narwhal
[2105.11827] Narwhal and Tusk: A DAG-based Mempool and Efficient BFT Consensus) we are also looking at integrating a new
virtual machine (move vm
https://developers.diem.com/papers/diem-move-a-language-with-programmable-resources/2019-06-18.pdf)
to allow scaling of payment transactions.

Other things that we could look into are improving the light client experience
and adding plumo sync (fast zero knowledge sync mechanism).

1 Like

Not just on the Celo network, but on other networks, people repeatedly ask about how to easily run their own full or archive node.

What would be useful is if Celo blockchain team included in a separate repo a set of maintained docker-compose stack configuration files (for full node vs archive node) and documented how to use the compose stack (with configurable env variables and such).

I think I answer the same questions about running the block chain docker client all the time and having a single official compose stack repo to point to would ease the adoption curve.

It would also be useful to include in this repo information about recommended cloud providers & machine configurations for each type of node to run.

5 Likes

Having more regular merges to stay closer to upstream head would be useful.
For example, the entire tracer package in Celo blockchain client is outdated compared to upstream and I had to manually port everything over from upstream.

I’m hoping that the next big upstream merge will bring over the new tracer modules from upstream and I can remove some of my own forked code.

3 Likes

Absolutely, we need an official repo providing docker configuration and documentation about the different types of nodes. We could raise issue and have a proper log of change there, at the moment the Run a Celo Full Node page is sometimes modified silently, the setup proposed is not necessarily fitting the needs.

kewl

Hi @kewl @diwu1989,

Thank you for your input on this, I agree that its not clear how to run different types of node and docker compose seems like a great way to encapsulate this information and make running a node simpler.

@diwu1989 why did you specify that this should live in a separate repo? My first inclination would be to add it to the celo-blockchain repo since this would simplify keeping it up to date with the implementation and facilitate running the currently checked out commit as well as released versions.

Hi @diwu1989, more regular merges is something we are probably not going to start doing right now. The reason being that merging is incredibly costly for us in terms of man hours, due to the divergence between our codebase and geth. As time goes on that divergence increases and so does the cost to merge. And although merging provides some valuable features, there is often a lot of stuff that is not particularly relevant to our immediate goals. This doesn’t mean that we are not going to pull important features from upstream, just that we may decide to pick and choose the features that are useful and port them across without merging everything up to that point. We are currently finishing up a merge which will take us up to upstreams 1.10.9.

With respect to tracing, what are the changes you wanted? This could be something that we could port outside of the normal merge process.

There’s 2 choices:

  • make an operations folder in the monorepo and keep it there
  • make a separate operations repo

Pro of mono repo is everything is together for the devs
Con of mono repo is that when someone else (operator user) want to spin up the stack, they’re cloning one giant ass mono repo…

Optimism for example uses on giant mono repo with operation infra folders, and in order NOT to clone the giant mono repo (cus i dont care about the rest of the repo), I literally copy paste the folder out to my own operational repo for deployments.

Tradeoff of developer friendly, vs operator friendly.

Basically everything from this subtree of upstream, I ported over already to my private fork of Celo:

They recently added in the Goja runtime, which i feel might be a pain to port…
Personally, I’m fine with you guys NOT dealing with this, cus I already did all this work myself privately.

What’s available right now in Celo Geth gives enough functionality for dev flow. That’s good enough for everyone.
The upstream has optimizations that make the tracers faster, fast enough to build generalized backrunners, which is what arb operators use. You don’t have to port this over, cus at this point, it won’t benefit the general dev population

My point was more about getting general quality of life features from upstream into Celo Geth faster to keep up with general changes there.