Posts

Billion-parameter LLMs are trained on what could be described as the whole Internet. Every published book, every blog post, white paper, Wikipedia page, Reddit post has been scraped and used for training. This represents trillions of data points, covering a broad range of subjects, including what humanity is, what humanity thinks, and how humanity thinks. With such an enormous amount of training data, it’s natural to ask whether LLMs possess a general understanding of human social behaviour, and, if so, whether we can use them as proxies to predict what an individual or even a specific subpopulation would do....

Intro It’s easy to share wins, motivational quotes, or shitposts. Today, I want to do something much harder: talk about failure. In a world full of polished success stories, fake growth hacks, and algorithm-chasing content, I thought sharing something raw and real might be more valuable. Not a launch post. Not a victory lap. Just a brutally honest check-in. Solight, my startup, isn’t going well. And I don’t know if I should keep going....

Introduction Every morning, I have to step over this same homeless person to get into my Waymo Those are my exact words, and I realize today, how well they describe San Francisco. It’s a troubling mix of richness, innovation, but also ugly despair. And despite all, I’ve never felt more right at home, than in SF. I target this blog post to Europeans, who, like me, might be dreaming of San Francisco, wondering what life is like over there, asking themselves if it’s the right place to be, worth crossing the Atlantic Ocean, or even if it’s like they heard it was....

Some forewords I’m currently experimenting with different ways to produce content in the hope to be able to produce more than one blog post a year in 2024. Expect some variations in my content style and feel free to share any advices or feedbacks you could have. Editing Models With Task Arithmetic Recently I came across mergekit a “tools for merging pretrained large language models”, this picked my curiosity about how exactly does one create a new model by “merging” or “fusing” multiple models into one....

A week ago I’ve received a ping from my CEO invinting me to join a meeting. No clue about the meeting’s subject nor the attendants. But an hour later, here I am joining a call with Gautier Soubrane and Maxime Portaz from Graphcore, delving into a topic entirely unfamiliar to me: IPUs. This meeting really piqued my curiosity. What in the world are IPUs? How are they different from CPUs or GPUs?...

Today I’m glad to introduce you to one of my favorite topics, the “adversarial examples” 🤠 Adversarial machine learning As cyber-security is the practice of attacking and protecting computing systems from digital attacks, adversarial machine learning (AML) is the study of the attacks on machine learning algorithms and the defenses against such attacks. These attacks can target different types of Machine Learning (ML) systems, such as escaping spam filters by obfuscating spam messages, misleading an intrusion detection system (IDS) by modifying the characteristics of a network flow or even leading an autonomous car to misinterpret a stop sign for a speed limit sign....