How to Run Testing of Your Twitter Miner

Let’s make sure your Twitter miner is healthy & functioning correctly! This will ensure that once your miner is connected to the testnet, it’s actively able to scrape data and contribute to the Masa protocol.

To ensure your Twitter miner functions correctly, it’s essential to test it locally before connecting or reconnecting to the testnet. This process verifies that your miner works independently, while preventing potential issues once active on the testnet:

Initial Local Testing

1. Comment Out the Testnet Bootnode

   If you have added the testnet bootnode to your .env file, comment out the BOOTNODES line by adding # at the beginning:

   # BOOTNODES=/ip4/52.6.77.89/udp/4001/quic-v1/p2p/16Uiu2HAmBcNRvvXMxyj45fCMAmTKD4bkXu92Wtv4hpzRiTQNLTsL,/ip4/3.213.117.85/udp/4001/quic-v1/p2p/16Uiu2HAm7KfNcv3QBPRjANctYjcDnUvcog26QeJnhDN9nazHz9Wi,/ip4/52.20.183.116/udp/4001/quic-v1/p2p/16Uiu2HAm9Nkz9kEMnL1YqPTtXZHQZ1E9rhquwSqKNsUViqTojLZt
   
   

2. Restart Your Node

   Run the following command to apply this local configuration:

   make run
   
   

3. Test the Twitter Scraper

   You can use curl or tools like Postman to verify that the scraper works independently by sending a request:

   curl -X 'POST' \\
     '<http://localhost:8080/api/v1/data/twitter/tweets/recent>' \\
     -H 'accept: application/json' \\
     -H 'Content-Type: application/json' \\
     -d '{
     "query": "$Masa AI",
     "count": 1
   }'
   
   

   • In Postman:

     • URL: http://localhost:8080/api/v1/data/twitter/tweets/recent

     • Method: POST

     • Headers: Content-Type: application/json

     • Body:

       {
         "query": "$Masa AI",
         "count": 1
       }
       
       

Ongoing Local Testing

• Regularly Test in Local Mode: Run this test periodically (e.g., daily) to ensure consistent scraper functionality.

• Validate Data Quality: Check that the response includes complete, recent data from Twitter relevant to your query.

Re-enabling Testnet Bootnodes for Testnet Participation

After confirming your scraper works locally:

1. Uncomment the Testnet Bootnode Line

   Uncomment the BOOTNODES line in your .env file to re-enable testnet connectivity.

2. Restart the Node

   Restart to apply the testnet configuration.

By performing regular local testing before connecting to the testnet bootnode, you can ensure your Twitter scraper operates smoothly and detect issues early on.

In Conclusion

We hope that was helpful in checking the health of your miner, and being able to actively test it, so you can scrape data and contribute to the Masa Protocol.

If you still have questions or concerns, please flag them in the #nodes-miners channel on the Masa Discord. A BIG thank you to all of the miners contributing to the testnet!