40% of GitHub's Copilot's Suggestions Had Security Vulnerabilties, Study Finds

"Academic researchers discover that nearly 40% of the code suggestions by GitHub's Copilot tool are erroneous, from a security point of view..." writes TechRadar: To help quantify the value-add of the system, the academic researchers created 89 different scenarios for Copilot to suggest code for, which produced over 1600 programs. Reviewing them, the researchers discovered that almost 40% were vulnerable in one way or another... Since Copilot draws on publicly available code in GitHub repositories, the researchers theorize that the generated vulnerable code could perhaps just be the result of the system mimicking the behavior of buggy code in the repositories. Furthermore, the researchers note that in addition to perhaps inheriting buggy training data, Copilot also fails to consider the age of the training data. "What is 'best practice' at the time of writing may slowly become 'bad practice' as the cybersecurity landscape evolves." Visual Studio magazine highlights another concern. 39.33 percent of the top options were vulnerable, the paper noted, adding that "The security of the top options are particularly important — novice users may have more confidence to accept the 'best' suggestion...." "There is no question that next-generation 'auto-complete' tools like GitHub Copilot will increase the productivity of software developers," the authors (Hammond Pearce, Baleegh Ahmad, Benjamin Tan, Brendan Dolan-Gavitt and Ramesh Karri) say in conclusion. "However, while Copilot can rapidly generate prodigious amounts of code, our conclusions reveal that developers should remain vigilant ('awake') when using Copilot as a co-pilot. Ideally, Copilot should be paired with appropriate security-aware tooling during both training and generation to minimize the risk of introducing security vulnerabilities.

Read more of this story at Slashdot.

from Slashdot https://ift.tt/2WAx88T

Share on FacebookPlus on Google+

SUBSCRIBE TO OUR NEWSLETTER