Hi all,

Had a small chat on #AI with somebody yesterday, when this video came up: "10 Things They're NOT Telling You About The New AI" (*)

What strikes me the most on this video is not the message, but the way it is brought. It has all the prints of #disinformation over it, .. especially as it is coming from a youtube-channel that does not even post a name or a person.

Does anybody know this organisation and who is behind it?

Is this "you are all going to lose your job of AI and that's all due to " message new? What is the goal behind this?

(Sorry to post this message here. I have been looking for a lenny/kbin forum on disinformation, but did not find it, so I guess it is most relevant here)

(*) https://www.youtube.com/watch?v=qxbpTyeDZp0

Anyone else getting tired of all the click bait articles regarding PoisonGPT, WormGPT, etc without them ever providing any sort of evidence to back up their claims?

They’re always talking about how the models are so good and can write malware but damn near every GPT model I’ve seen can barely write basic code - no shot it’s writing actually valuable malware, not to mention FUD malware as some are claiming.

Thoughts?

Callisto is an intelligent automated binary vulnerability analysis tool. Its purpose is to autonomously decompile a provided binary and iterate through the psuedo code output looking for potential security vulnerabilities in that pseudo c code. Ghidra's headless decompiler is what drives the binary decompilation and analysis portion. The pseudo code analysis is initially performed by the Semgrep SAST tool and then transferred to GPT-3.5-Turbo for validation of Semgrep's findings, as well as potential identification of additional vulnerabilities.

This tool's intended purpose is to assist with binary analysis and zero-day vulnerability discovery. The output aims to help the researcher identify potential areas of interest or vulnerable components in the binary, which can be followed up with dynamic testing for validation and exploitation. It certainly won't catch everything, but the double validation with Semgrep to GPT-3.5 aims to reduce false positives and allow a deeper analysis of the program.

!

A decompiler-unified plugin by Zion Basque that leverages the OpenAI API to enhance your decompilation process by offering function identification, function summarisation and vulnerability detection. The plugin currently supports IDA, Binja and Ghidra.

!

Codex Decompiler is a Ghidra plugin that utilizes OpenAI's models to improve the decompilation and reverse engineering experience. It currently has the ability to take the disassembly from Ghidra and then feed it to OpenAI's models to decompile the code. The plugin also offers several other features to perform on the decompiled code such as finding vulnerabilities using OpenAI, generating a description using OpenAI, or decompiling the Ghidra pseudocode.

In this post, I introduce a new Ghidra script that elicits high-level explanatory comments for decompiled function code from the GPT-3 large language model. This script is called G-3PO. In the first few sections of the post, I discuss the motivation and rationale for building such a tool, in the context of existing automated tooling for software reverse engineering. I look at what many of our tools — disassemblers, decompilers, and so on — have in common, insofar as they can be thought of as automatic paraphrase or translation tools. I spend a bit of time looking at how well (or poorly) GPT-3 handles these various tasks, and then sketch out the design of this new tool.

If you want to just skip the discussion and get yourself set up with the tool, feel free to scroll down to the last section, and then work backwards from there if you like.

The Github repository for G-3PO can be found HERE.

LLDB plugin which queries OpenAI's davinci-003 language model to speed up reverse-engineering. Treat it like an extension of Lisa.py, an Exploit Dev Swiss Army Knife.

At the moment, it can ask davinci-003 to explain what the current disassembly does. Here is a simple example of what results it can provide:

!

IDAPython script by Daniel Mayer that uses the unofficial ChatGPT API to generate a plain-text description of a targeted routine. The script then leverages ChatGPT again to obtain suggestions for variable and function names.

Gepetto is a Python script which uses OpenAI's gpt-3.5-turbo and gpt-4 models to provide meaning to functions decompiled by IDA Pro. At the moment, it can ask gpt-3.5-turbo to explain what a function does, and to automatically rename its variables.

This is a little toy prototype of a tool that attempts to summarize a whole binary using GPT-3 (specifically the text-davinci-003 model), based on decompiled code provided by Ghidra. However, today's language models can only fit a small amount of text into their context window at once (4096 tokens for text-davinci-003, a couple hundred lines of code at most) -- most programs (and even some functions) are too big to fit all at once.

GPT-WPRE attempts to work around this by recursively creating natural language summaries of a function's dependencies and then providing those as context for the function itself. It's pretty neat when it works! I have tested it on exactly one program, so YMMV.

A very interesting approach. Apparently it generates lots of results: https://twitter.com/feross/status/1672401333893365761?s=20

They used OpenSSF Scorecard to check the most starred AI projects on GitHub and found that many of them didn't fare well.

The article is based on the report from Rezilion. You can find the report here: https://info.rezilion.com/explaining-the-risk-exploring-the-large-language-models-open-source-security-landscape (any email name works, you'll get access to the report without email verification)

All of these might not work as well anymore, but they're still interesting to take a look at.

As they mention in the thread, this isn't exactly groundbreaking but it's still interesting.

> Strong preference will be given to practical applications of AI in defensive cybersecurity (tools, methods, processes). We will grant in increments of $10,000 USD from a fund of $1M USD, in the form of API credits, direct funding and/or equivalents.

I think this is a great initiative and I hope we'll see some cool projects to benefit defenders.

cross-posted from: https://feddit.it/post/352229

> After being scammed into thinking her daughter was kidnapped, an Arizona woman testified in the US Senate about the dangers side of artificial intelligence technology when in the hands of criminals. > > Jennifer DeStefano told the Senate judiciary committee about the fear she felt when she received an ominous phone call on a Friday last April. > > Thinking the unknown number was a doctor’s office, she answered the phone just before 5pm on the final ring. On the other end of the line was her 15-year-old daughter – or at least what sounded exactly like her daughter’s voice. > > “On the other end was our daughter Briana sobbing and crying saying ‘Mom’.” > > Briana was on a ski trip when the incident took place so DeStefano assumed she injured herself and was calling let her know. > > DeStefano heard the voice of her daughter and recreated the interaction for her audience: “‘Mom, I messed up’ with more crying and sobbing. Not thinking twice, I asked her again, ‘OK, what happened?’” > > She continued: “Suddenly a man’s voice barked at her to ‘lay down and put your head back’.” > > Panic immediately set in and DeStefano said she then demanded to know what was happening. > > “Nothing could have prepared me for her response,” Defano said. > > Defano said she heard her daughter say: “‘Mom these bad men have me. Help me! Help me!’ She begged and pleaded as the phone was taken from her.” > > “Listen here, I have your daughter. You tell anyone, you call the cops, I am going to pump her stomach so full of drugs,” a man on the line then said to DeStefano. > > The man then told DeStefano he “would have his way” with her daughter and drop her off in Mexico, and that she’d never see her again. > > At the time of the phone call, DeStefano was at her other daughter Aubrey’s dance rehearsal. She put the phone on mute and screamed for help, which captured the attention of nearby parents who called 911 for her. > > DeStefano negotiated with the fake kidnappers until police arrived. At first, they set the ransom at $1m and then lowered it to $50,000 when DeStefano told them such a high price was impossible. > > She asked for a routing number and wiring instructions but the man refused that method because it could be “traced” and demanded cash instead. > > DeStefano said she was told that she would be picked up in a white van with bag over her head so that she wouldn’t know where she was going. > > She said he told her: “If I didn’t have all the money, then we were both going to be dead.” > > But another parent with her informed her police were aware of AI scams like these. DeStefano then made contact with her actual daughter and husband, who confirmed repeatedly that they were fine. > > “At that point, I hung up and collapsed to the floor in tears of relief,” DeStefano said. > > When DeStefano tried to file a police report after the ordeal, she was dismissed and told this was a “prank call”. > > A survey by McAfee, a computer security software company, found that 70% of people said they weren’t confident they could tell the difference between a cloned voice and the real thing. McAfee also said it takes only three seconds of audio to replicate a person’s voice. > > DeStefano urged lawmakers to act in order prevent scams like these from hurting other people. > > She said: “If left uncontrolled, unregulated, and we are left unprotected without consequence, it will rewrite our understanding and perception what is and what is not truth. It will erode our sense of ‘familiar’ as it corrodes our confidence in what is real and what is not.”

This was better than I expected. A broad overview of how they approach red teaming AIs, rather than specific "look at this one prompt injection" which makes it more valuable long term.