American artificial intelligence firm Anthropic – which has developed a family of large language models called Cloud – has created a unique position in their firm.
Amanda Askell says the models have a “human-like element”, adding that they will “inevitably form a sense of self.” (askell.io)The role is occupied by Amanda Askell, the resident philosopher, who is tasked with creating the personality of the Claude AI and helping to give it a moral compass so it can distinguish morally right from wrong.
Askell, who wanted to be a philosopher since he was 14, had to learn Claude’s reasoning patterns and talk to the model, which creates the AI’s personality and corrects its mistakes with prompts that can run for more than 100 pages, the Wall Street Journal reported.
The 37-year-old philosopher says models have a “human-like element,” adding that they will “inevitably form a sense of self.” In his biography, Askell says he is a philosopher and working on fine-tuning and AI alignment at Anthropic.
He says his team trains models to be more honest and develop “good character traits”. Askell, who completed his PhD in philosophy from New York University (NYU) and his Bphil in philosophy from the University of Oxford, previously worked at OpenAI as a research scientist in the policy team.
He was responsible for working on “debating AI safety and human baselines for AI performance”.
What does Ascale work at Anthropic include?According to WSJ, Anthropic compares his work to the effort involved in raising a child, as he trains the cloud to distinguish between right and wrong by giving it unique personality traits.
The resident philosopher instructs the AI to read subtle cues to help develop its emotional intelligence. Askell further develops Claude’s understanding of himself to prevent him from being easily manipulated, and from seeing himself as anything other than helpful and humane. Simply put, her job is to teach Claude to “be better,” the WSJ reported.
Askell, who is originally from rural Scotland, said he would keep the AI models under control despite occasional failures. Philosopher, who has defensiveness when it comes to Claude, says we’d do well to treat them more sympathetically. Askell believes this is crucial because it’s not possible for Claude to have real feelings, but rather how he says how we interact with AI will shape what they become.
Askel created Claude’s ‘soul document’Askell drafted Claude’s new ‘Constitution’ or its ‘Document of the Spirit’, which was recently released for public viewing.
The Constitution straddles the line between a moral philosophy thesis and a company culture blog post, Time magazine reported. The document is addressed to Claude, and is used to develop the model’s character during various stages of training. It directs AI to be safe, ethical, adhere to anthropic guidelines and be helpful to the user.
Askell also believes that it can be beneficial for language models to explain why they should behave in certain ways. “Rather than just saying, ‘Here’s a bunch of behaviors we want,’ we’re hoping that if you give the model reasons why you want these behaviors, it’s going to generalize more effectively to new contexts,” Time magazine quoted him as saying.
