A recent experiment by Coinbase has sparked debate in the crypto community about whether artificial intelligence (AI) can replace human-led audits in ensuring the security of token transactions.
Despite some promising results, there is still a need for improvement, as the research shows that ChatGPT isn’t a capable replacement for human audits.
Coinbase Experiments With AI To Determine The Accuracy Of Its Security Reviews
Coinbase recently employed artificial intelligence to evaluate the accuracy of its security reviews of tokens listed on the exchange. With the help of ChatGPT (the AI tool), the leading exchange tested twenty smart contracts and matched its results with the manual review results twelve consecutive times.
Unfortunately, five cases were missed by ChatGPT, with the worst failure being a high-risk asset incorrectly labeled as low-risk. The trial showed that sometimes the AI-generated inconsistent results, with the same data input resulting in different outputs, particularly when shifting from one ChatGPT version to the next.
Nevertheless, the Coinbase team hopes that with additional data engineering, there would be an improvement in the precision of ChatGPT such that it can be utilized as a secondary quality control measure. A spokesperson from BlockSec, a blockchain security infrastructure firm, revealed that they are not astonished as other traditional programming tools can audit smart contracts automatically.
Yet, they remarked that the AI could not be effective for complex business logic, the primary attack surfaces, and gaps that smart contract audits should concentrate on. Coinbase expressed confidence that the tool could provide further safeguards.
Other security professionals in the cryptocurrency security arena agreed with this submission. At present, Artificial Intelligence is not a substitute for a human being.
However, the research notes that it is a huge help, especially for those who are fatigued or need to pay attention.
Why Chatbots AIs can’t replace Crypto Audits
Understanding the process of auditing a token or coin is essential to understand why chatbots can’t replace human audits. Audits involve several steps, including code review, security checks, and other assessments.
These assessments are conducted by skilled professionals who must understand the complexities of the blockchain and the underlying technology. By contrast, a chatbot is only as good as the data feed.
Hence, it needs to have the expertise or experience of a human auditor. Therefore, it can’t replace the role of a human auditor.
This is especially true when it comes to assessing the security of a blockchain. Nevertheless, the Coinbase experiment showed that chatbots could be valuable for auditing tokens.
Until chatbots can understand the complexities of the blockchain and the underlying technology, they will remain an invaluable asset but not a viable replacement.
