Imagine that you want to check if the web3 service is private or not. But you aren’t a protocol architect or core contributor. Where would you start?
The Web3privacy now community proposes to use this simple & actionable playbook for techies. It helps to:
perform check-ups if projects claiming privacy features are legit
filter out high-risk services
boost web3, privacy & security knowledge base
empower yourself with an actionable dev toolkit
junior or general devs with under or around 1-year coding experience
with poor security auditing skills
who are interested in privacy tech
The general public within the devs landscape is the most vulnerable audience compared to mature developers. Generalisation helps to prototype universal tooling & secure research from the niche approaches. And to build the right skills for resilient privacy tech.
Audit check-up (manual)
Dev toolkit DYOR
devs lack the basics of security audits (while auditing companies or security engineers create encyclopedias that require lots of dev hours to read & more to understand)
case studies could help to "visualize" the assessment journey
DB of the malicious practices & critical bugs within the privacy landscape will come in handy for practical devs
Hypothesis: can the market come up with a standard of privacy features check-up across multiple projects that will increase general privacy stack levelling?
Crucial scoring context: time on research x potential amount of privacy anti-features detection. This means that scoring could be scalable if more people use it in the economy, and general business & without expert knowledge level.
We take the scoring model generated for non-techies.
Add dev-centric modules, tools & journeys.
Generate 2in1 model (incl non-techies & techies in 1).
The scoring model for non-techies “Validity track” covers GitHub, Product-readiness, Team, Docs, Audit.
Note: quick assessment helps to decrease privacy dark patterns from obscure language to test-net claiming it has a "state of art privacy".
Privacy is a complex web3-industry covering hundreds of use cases & general crypto stack features. Meaning that there’s an embedded difference in audit complexity between Brume Wallet & DarkFi L1 protocol.
We don’t expect developers to become pen-testers after reading the scoring model & audit L1s with ease. Increasing privacy check-up awareness among devs & moving to standards for privacy awareness - tangible goal.
Again, keep in mind market complexity
Diverse use cases: "transaction", "storage", "messaging" etc.
Specific privacy features (from use case to use case): "sender & receiver are un-linked within the transactional privacy" or "content of the message is End-to-end encrypted within the Messenger privacy".
“Use cases stack”: many projects communicate complex privacy: L1, L0 (sharing multiple privacy categories & features. Examples: Nym - network-level privacy (incl metadata protection); DarkFi - anon DAO, anon smart contracts & multiple other privacy-enhancing protocol features); or even Railgun sharing chain-specific (Ethereum, Polygon) or asset-centric (NFT) privacy.
Is it in stable release, 1.0 and not an alpha or untested code?
Are there many PRs and Issues pending?
Are there external contributors outside of the team members? example
What are licenses in use? Privy Apache-2.0 license
example: the hoprd client is released under GPLv3 license that allows you to do that (and also modify and re-distribute) Hoprn freely
observation: devs can observe not only the liveliness of GitHub but also the quality of the repos & contributors’ profiles (from historical track record to reputation).
read the documentation:
Is it comprehensive?
How well-written are privacy & security assumptions/guarantees?
If applicable: how well-written is the encryption method? example
Is whitepaper marketing or dev-centric?
observation: devs can read in-depth descriptions of the privacy specs & test dev-centric docs.
Check if there are known contributors (reputation 101)
Check commits at GitHub
How many community contributors beyond the core team?
How many technical specialists are in the team?
How mature are core contributors (previous projects, GitHub commits)?
observation: devs can dive deeper into tech x non-tech team structure, public reputation, track record behind CTO & core devs.
Were bugs fixed? Zokyo x Railgun_ example, p.7
How reputable is an audit company (manager who performed the audit)?
observation: devs can trace critical bug resolution; relation between vulnerabilities & historical improvements.
Where are the nodes (check block explorer) Nym mainnet explorer
Number of nodes (the larger the footprint the best privacy)
observation: devs can audit note footprint, check up market benchmarks, and analyze geography & type of node runners (VC, community etc).
Check the dependencies: more dependencies mean more risk of security vulnerabilities and more risk of network code that can leak your IP or personal data
Read terms and conditions of every third-party dependency, its licenses and what network calls are made by simply adding the library to the project even when not being used
Do they have a third-party audit?
What's the reputation behind the library team? What's the reputation behind security audit company?
example: Rabby - 1150 dependencies, Metamask - 720, Brume - 50 link
observation: devs can audit third-party dependencies increasing vulnerability risks, check up on these libraries’ past exploits, necessity of usage etc.
No email or tel number for signup
Does not implement KYC or AML
What user information is stored? (username, IP address, last connection, wallets associate, etc)
Amount of transactions (Dune, DeFi Lama, block explorer etc)
Number of people using it
DAO structure (if applied)
How centralized is the protocol governance? Railgun_ governance docs
The updated model consists of GitHub, Product-readiness, Team, Docs, Audit, Contributors, Licenses, Support, Dependencies & Data aggregation section.
It helps developers improve the “Validity track” privacy assessment & decrease potential privacy dark patterns.
This model is a current MVP for techies that requires qualitative & quantitative field testing - this will be covered in the forthcoming Part 2 of the Playbook delivery on Mirror.
Apply to the practical examples listed below.
example 1: Railgun
Use @Railway_xyz to send a private transfer to a 0zk address through a Relayer.
Examine the "receipt" of that transfer on etherscan or arbiscan.
You will not find:
amount anywhere in the transaction receipt.
This is a 0zk -> 0zk transaction (sending tokens from one #DeFi user to another. You'll note that the scan has a from address but this is simply a Relayer address that pays gas to process the on-chain computation).
User communications with a Relayer are passed via through the @waku_org p2p gossip network, so Relayers can't know a particular message origin.
In other words, even here privacy was a big consideration throughout the tech stack not just on Etherscan.
The To address is simply the RAILGUN smart contract on @0xPolygon in this case. So
nothing is revealed about the recipient. The value that exchanged hands? Private.
try to decode the input data for the transaction? It's all encrypted.
So how much money exchanged hands here? Well, the short answer here is - it's private. Only the sender and recipient will know. There will be zero-knowledge about it unless they choose to reveal the transaction information.
example 2: HOPR
The only user of HOPR is actually @RPC_h_- so I'd check if that is private.
Use DERP in your normal wallet.
See how every request from your wallet gets disclosed via the website.
Then install RPCh.
Connect it to the DERP endpoint
And see that no request is visible on the page anymore, because the requests are sent via other IP addresses.
example 3: dm3
The criteria for a messenger or messaging protocol/service to prove "privacy by design":
Decentralized delivery network (no central instance controls the flow/storage/transmission of messages or identity information)
Decentralized registry for communication info (no entity controls the information how to encrypt and deliver messages)
End-2-End Encryption, ideally with PFS (every message is end-to-end encrypted, ideally with rotating keys (PFS, ...))
Messages are not stored in the network (web3 storage like IPFS-based storage is critical as we never know if the encryption will be secure in the future, if not, this info is public)
No traceable transmission (transmission of messages must be not traceable from outside, as this may reveal the connection)
Open-source of security-relevant parts (closed systems can't be trusted to not have backdoors)
example 4: MASQ
Have you tried the MASQ Browser
Run a cli node on a VPServer
Read the docs on docs.masq.ai
Break TLS at first as there is a TLS connection from the MASQ Browser to the destination web server.
example 5: Penumbra
Run a node and see what data is synced from the network.
Check out the various RPCs like get block by height and see that the transaction data is encrypted (aside from what is intended to be public like fees).
Look in the local pcli database and see that only your transactions are present in decrypted form.
Verifying the confidentiality/integrity of the encryption is its own whole topic that requires cryptographic expertise.
Ehe RPC endpoints use TLS and our RPCs speak in protobuf gRPC. So you'll need to decode the TLS and decode the gRPC protobuf messages.
You might have an easier time making custom builds of pd and cometbft to dump the data they receive.
Getting a basic understanding of how ABCI works with cometBFT will help you with this a lot. They have pretty good docs on the cometBFT website.
Here is the pd code that receives and processes ABCI messages from cometBFT. You could add debugging code here to dump out the contents of the ABCI messages: https://github.com/penumbra-zone/penumbra/blob/main/crates/core/app/src/app/mod.rs
example 6: OMNIA
off-chain privacy - mostly protecting metadata like the IP address of the one making the request. Algorithm to validate this:
Set up a dummy RPC node (i.e. using netcat or anything),
Register it in a load balancer in OMNIA.
Make requests to OMNIA LB endpoint.
The expected result is that you will not see your original IP in the data you receive on the dummy node, but rather random proxy addresses used to protect customer data
Private transactions are also known as transactions sent through private mempool. Algorithm to validate this (for now only mainnet):
Send a transaction through a frontrunning protected endpoint.
Look at the transaction with a third-party explorer like Etherscan.
The expected result is that you'll see a label of "MEV Transaction"/"Private Transaction" which usually means the tx was sent through a private mempool, hidden from the bots etc that are eyeing the public mempool: Example.
example 7: Firn
Look at an existing withdrawal on Etherscan.
See if you can understand why the identity of the withdrawer is hidden.
See if you can understand what information is visible (zkp, timing, withdrawal amount, etc.). Example: etherscan
Skim through the code of all of our contracts, especially Firn.sol:
example 8: Waku
Read the doc to understand at a high level
Read the specs to understand exact behaviour and limitations
Run code and look at the network/libp2p logs:
What node it's connecting too?
What messages are being sent?
example 9: Leo wallet
Ovro Ahmed | Leo wallet
For a developer conducting a privacy-service self-audit, a simple algorithm could involve
checking Leo's code repository for any private or sensitive data-handling functions
inspecting API endpoints for encryption
and reviewing documentation for privacy features.
Utilizing tools can enhance the assessment:
static code analysis and
For static code analysis:
the link you provided is a helpful resource: https://owasp.org/www-community/controls/Static_Code_Analysis
As for privacy-focused libraries (widely used and prioritizing user privacy and data protection):
network requests: Wireshark on the Docker container to check what it interfaces &/or Console - check network requests by opening the console, the more requests it does to external servers, the more your IP address is sent around
transactional analytics: Etherscan
node connections: netstat would be enough to see the connections made by the node
Android: feel free to test it at Exodus Privacy - a friendly service to investigate Android apps. Check https://f-droid.org and verify the “no anti-features” tag (like dependency on third-party services): Example
Exact # of transactions (Dune, DeFi Lama, block explorer etc): min to reflect liveliness
Exact # of people using it: min to reflect liveliness
Exact # of dependencies (Rabby - 1150 dependencies, Metamask - 720, Brume - 50): critical impact on potential privacy exploitations
Where are the nodes (check block explorer: the bigger diversity - the better).
Number of nodes (the larger the footprint the best privacy).
DAO structure (if applied)
How centralized is the protocol governance?
We will cover quantitative & qualitative scoring in the forthcoming Part 2 of the scoring model. Engage more privacy devs & project leaders to reflect on the assessment journey & empower existing take.
Source repo on Git is available for all curious minds & those who want to contribute & improve our model