@shadowwwind all three of them were reported by people with knowledge, respect and understanding. Just proving that tooling is still just one part of this.
@bagder@shadowwwind This is good to see - AI still produces a lot of crap and loves to randomly change code - but when focused on finding issues with a human who understands the code I've found the hit rate is close to 100% correct so far.
@tanepiper@shadowwwind@bagder I suppose using AI to find vulnerabilities is not unlike using a fuzzer to find vulnerabilities. Sure, it can find many positives, but it's up to the human to determine if it's a true or false one.
@sijmen@shadowwwind@bagder My concern here that LLMs do give worse-than results than something like Sonar - in many cases static code analysis was good enough to catch things that LLMs miss.
Ideally we would still use different tools, but GitHub seems to want to push everything into models.
@tanepiper@shadowwwind@bagder I definitely agree that LLM vulnerability finding should only be done after the right tools are in place, like, indeed, static analysis
