Twitter: @soniajoseph_

Contents

1. Intro: Experimenting with mirror.xyz
2. Section 1: Overview of My Mind, & Thinking about Intelligence
   1. My Mind is a Two-State Dynamical System
   2. Intelligence as Feedback Loops
   3. Mila as an Institution
   4. Bay Area/NYC-Montreal Arbitrage
3. Section 2: Founding Companies in Web 3
   1. Founding in Web 3
   2. Web 3 vs Web 2 Investors
4. Section 3: Movement & World Building
   1. Metaphysics Tycoon / “New Transhumanism”
   2. New AI Safety Aesthetics
5. Section 4: Mathematical Thinking & Graduate School
   1. Math as Self-Help
   2. A Well-Factored Mind
   3. Frontier Research vs. First Principles
   4. Mathematician vs. Founder Thought Patterns
6. Outro: Long-term Vision

Experimenting with mirror.xyz

Mirror.xyz recently opened its doors, and so this post is an experiment in new publishing mediums. I am testing many platforms over the next few months, from mirror.xyz, to Twitter, to Medium, to Typeshare, and to services yet uncovered.

I am especially interested in the links between web 3 and publishing, so if you uncover people building interesting infra, do get in touch.

I’m not sure whether I’ll keep writing on Mirror.xyz, or switch to something else during the next few months of experimentation. There’s something deeply satisfying about this platform’s minting capabilities and the financial links to the Ethereum network. However, you cannot comment on this post yet, so Twitter remains necessary (but you can send me NFTs and ETH lol). I included an embedded NFT for early readers for some metaphysics fun.

The rest of this post is a forest of concepts: founding in web 3, movement-building, and mathematics as a world view. I’ll eventually dismantle this post into smaller posts and Tweet threads, in the style of 2022 digital writing. But for now, I like the breadth of the post, which feels like a chapter of a novel.

Regardless whether I keep writing on blockchain, let these thoughts remain eternal on Arweave, to be read by my descendants and their descendants after them.

Overview of my mind

My feedback loops turned my mind into a two-state dynamical system

For context, you can read October 2021 Update 1 written on my ancient, rusty GitHub, which details my recent move from Montreal to the Bay Area to research as a graduate student at Mila, the world’s largest academic institute in machine learning.

Since October, my life got fascinating very quickly. I am constantly bombarded by highly complex streams of information. My priorities remain becoming a world-class founder, researcher, and writer. By default, researchers do not make the best founders, and founders do not make the best researchers. Developing the skills to be good at both is time-consuming and difficult-- but it can be done, especially if you do transfer across domains.

My brain is a two-state oscillatory system, with one state in the artificial intelligence, and another state in web 3 founding. Traveling between New York, Montreal, Boston, and San Francisco over the past year, I appreciate how my personality and identity are a function of my feedback loops, and how these cities create who I am.

To make my feedback loops powerful and self-improving, I have aligned myself with communities who reflect the person I want to become. Mila (AI research), South Park Commons (SF/NYC founders, tinkerers), and Kernel.community (web 3) have been a sparkling synthesis of information streams. The trade-off is that I do not spend much time resting, or hanging out. I am constantly working. At this stage in my life, when I have no familial commitments, this is exactly what I want.

Something incredible is happening to my mind from being exposed to this level of information complexity. I’m not sure what it is, as my mind does not have enough bandwidth to model itself right in high resolution now, and I expect everything to be clearer in ~10 years. But it’s a very, very positive thing and my mind wants more of it. My mind feels like a system undergoing a phase transition.

The complexity of my information streams requires relentless focus and energy, especially when the distribution of my inputs is rapidly shifting.

I’ve devised several systems to handle this distributional shift. The geography of NYC-Montreal plays a key role in separating thought patterns. I feel like I’m slingshotting between the two cities in a figure-eight. I have slightly modified personalities and identities in each city, which the geography helps to organize. On certain days, I only do specific tasks: some days are devoted solely to research, others to company-building. Working at Mila, the physical institute, is also a useful tool for centralizing focus in AI. And finally, writing posts like this one aids integration.

Intelligence as feedback loops

Intelligence is made of feedback loops. Optimal control and dynamical systems understand this well, when you model the agent’s interaction with the environment as a system that can converge to a steady state. Reinforcement learning writes down intelligence emerging from feedback loops between an agent and its environment, and between competing and coordinating agents.

As one of many examples, companies are feedback loops: they are cybernetic collectives of humans and machines. Startups and their investors create each other, reminiscent of a GAN. Founders will iterate on a company, which must pass the investor’s discriminator: or vice versa, investors will create term sheets that must pass the founders’ judgement. Practically, even if your startup does not need money, many founders tell me that they take funding for the feedback loops with the investors, not for the cash itself. Obviously, there are degenerate cases of this as well.

While math represents intelligence as rising from feedback loops-- between people and their environment, between people and each other, we humans usually forget. Our study of intelligence has long been dominated by Western paradigms in which agents are clearly distinct from the environment, and intelligence is atomized in the brain. The cultural roots run deep in the West, especially in America, where lone genius, great man theory, Thoreau-in-the-cabin, Ralph Waldo Emerson, Nietzsche, and Ayn Rand myths still largely pervade.

I notice, however, that my own intelligence is incredibly conditioned on the intelligence of my environment. Often I do not feel like an intelligent being, but the medium for the intelligence that exists outside of me. In one viewpoint, I am simply the compiler of the intelligence streaming from the New York web 3 and Montreal AI research worlds. You can throw endless mathematical analogies onto intelligence: In another frame, intelligence is best represented as the large-scale dynamical system that is Earth-- a system that can be recursively decomposed into sub-systems.

Mila as an institution

As a disclaimer, my description of Mila is not to be taken comprehensively, because I have only been here for three months as a new graduate student.

So far, this place has exceeded expectations by a lot, and they were already quite high. Mila is incredibly diverse: there are many international researchers with different methodologies and viewpoints. The institute is non-hierarchical in culture, and the people are collaborative and friendly. I feel comfortable DM-ing anybody on Slack to discuss their research.

Unlike many other universities, distinctions between subfields do not prevent researchers from mingling with each other. Researchers from all subfields of AI, including NLP, CV, RL, neuroscience, and mathematics all know each other and socialize in one communal area. The structure of the institute, with one large coffee room in the center, allows for casual water cooler interactions.

Mila is also highly amenable to industry collaboration, with one building devoted to company and startup offices. My impression is that the institute has grown largely throughout the pandemic, and that the Canadian government is channeling funding into Mila with the intention of it becoming an economical and technological powerhouse.

Given that Mila seems highly diverse and there’s no one dominant viewpoint, I am viewing this place as a possible ground to develop alternative narratives to “the threat of superintelligence” and concepts of “AGI”.

Many of my posts and Tweets are reactionary to the “threat of superintelligence,” which was first proposed by Nick Bostrom. His thesis inspired cultural narratives-- and these narratives, in turn, drew hundreds of millions in capital into various subcultures, institutes, and funds-- from Less Wrong, to MIRI, to Open Philanthropy’s funds, to more mainstream research institutions like OpenAI and DeepMind. Even when not direct, Bostrom’s influence is felt deeply by the AI safety world.

Philosophy is clearly critical in creating the bedrock of many coordination structures. I’d like to create a body of AI philosophy that questions and decorrelates itself from Bostrom. The narrative would be equally ambitious in scope, but different in aesthetic. I write more about this later in this post, when discussing reworkings of transhumanism.

Bay/NYC and Montreal Arbitrage

I spent the first quarter of my life metalearning across complex environments, including the financial, academic, big tech, and startup worlds. The exploration was rather time and energy-intensive, but now I expect to see pay-offs.

There is an arbitrage opportunity between Bay Area / NYC capital and Montreal academic and technical knowledge, which could result in the creation of tech startups and private research organizations. There is further ideological arbitrage in bringing startup culture to Mila, and leveraging Mila’s immunity to the more homogenous AI narratives that I found in the Bay Area.

Founding in web 3 / AI

Founding a Web 3 Company

After intensive research throughout the past fall, I am sold on NFTs as a fundamental technology and building a company in the space.

The short story is that one of my web 3 side projects began ballooning. NFT.NYC energy and a reunion with a childhood friend launched into something that is gaining momentum. We realized we were creating something far larger than another NFT art drop and technical marketing scheme, something that could be fundamental to long-term infrastructure.

This company is the right alchemy of collaborators, market opportunity, and domain expertise, so I am pushing on it. It further leverages my background in NYC publishing / CS.

We’re raising in a few weeks but semi-stealth for now. You are welcome to get in touch if you’re interested in working with web 3 publishing models, or just to read the white paper.

Web 3 vs Web 2 Investor Differences

I notice interesting differences in fundraising in web 3 vs web 2. For web 3 investors, I notice less pushback to any uncertainty of the idea. This is partially due to the unpredictability of the crypto market, but also due to personality profile. Many of these investors were investing in Ethereum when ETH was at $100 or less, and have weathered through many bear-bull cycles. Cryptocurrencies losing $400 billion in market cap, as they have over the past two days, does not phase these investors.

This is not to stay that web 3 investors are less grounded than web 2 investors. Rather, web 3 investors seem to have a different relationship to uncertainty, a different set of epistemics, which allows the competent ones to thrive in uncertain environments.

The web 3 investing zeitgeist can be arbitraged into other emerging technologies. The growing number of longevity DAOs is no coincidence. Sectors working on difficult emerging technologies, such as brain-computer interfaces may find a stronger resonance with the risk profile of web 3 investors.

World Building, Movement Building

Metaphysics Tycoon & Transhumanism, Rebranded

Sometimes, when my mind boils with information, I forget the original reasons for taking actions, much like the protagonist of Memento.

Metaphysical conclusions drove my entry into the AI research and crypto/web 3 worlds in the first place. But it’s hard to keep metaphysical details in working memory while understanding the details of a new web 3 protocol or multi-agent AI system. The flood of technical knowledge makes me forget old parts of my identity, until I rederive these aspects of myself once more. For the protagonist of Memento, this is the function of tattoos: to create stable feedback loops in your thoughts, to prevent amnesia, and to remind yourself of versions of who you once were.

So what do I want?

A few people ask me why my interests sometimes seem so divergent. My answer is that they are not. My thought process is that humans are currently fumbling around in darkness, and we can reach a more enlightened state by augmenting our intelligence. Many people will immediately jump to psychedelics and BCIs, and yes, these are possible paths, but the scope is far greater and ties into economics.

For example, Neuralink is a very obvious and direct attempt to increase human intelligence. However, Neuralink is partially possible because the founder coordinated financial and human capital in the correct manner to build a machine. And the machine is not the BCIs the company is making, but the company itself.

Through startups and science, I would like to build more company-scale machines that question the nature of reality. And here’s where the formal rules for each sector break down, and you start operating across sectors to play a larger game, in which there are no rules (except moral and legal ones).

The comic version of my motivation is that I want to break out of the Matrix. The more grounded version is that I want to create industrial research and engineering bets that push at our conception of reality, in an effort that I term Metaphysics Tycoon.

So what is Metaphysics Tycoon?

Metaphysics Tycoon is the synthesis of science, economic coordination, and spirituality to gain a deeper insight into the nature of reality.

We are no longer restricted in 19th century gentleman science of a lone tweed-clad man meticulously cataloging pigeons (even though this is a cool image). Rather, we can understand the nature of reality, an ancient religious instinct, by forming large-scale coalitions of investors, researchers, engineers, and entrepreneurs. Our tools are not merely microscopes, but cybernetic machines of technology and people, bodies of institutes and companies working together.

Examples of Metaphysics Tycoon companies would include AI, longevity, space exploration, and genetics companies-- anything that falls under the old-school bucket of “transhumanism.” Neuralink, DeepMind, and OpenAI come up as obvious examples.

Metaphysics Tycoon is further predicated on the idea that most of our environment is uncertain. We do not know enough to be making strong philosophical or moral conclusions, so we should systematically search the space of possible outcomes. In accounting for my uncertainty, I take current moral frameworks less seriously as indicators of “truth.” Perhaps this is the greatest divergence from Effective Altruist frameworks, which put the reduction of suffering as a core axiom. While I see that reducing suffering is important, I am not convinced that it should be dogmatically held as a core goal-- rather, suffering is a poorly represented human concept.

You may ask why not just use the term “transhumanism,” and the answer lies in connotation. Transhumanism flourished in the 1990s with the extropians, so one simple argument is that the movement is simply thirty years out of date.

But there are also deeper reasons. Transhumanism implies that humanity must be transcended, which is an unnecessary connotation (rather, through technology, we can become even more human, instead of less). There are also ubermensch, Nietzsche, male, whiteness, fringy, and outcast connotations to old-school “transhumanism” that are not causal to its core mission, which I see as exploring the possibility space of existence itself.

So instead of attempting to salvage the word transhumanism, I am designing a new, less correlated aesthetic.

Below I have included a Metaphysics Tycoon NFT for the brave souls who have traversed this far into this post. If you are ready to play Metaphysics Tycoon, the infinite game, then do mint one. When I will notice it in your MetaMask wallet when we cross paths sixty years from now, our coordination will have a higher level of understanding.

Designing new AI safety aesthetics

Closely linked to transhumanism is the cultural movement and technical challenge of AI safety, or combating the threat of superintelligence. I would like to create something truly connotatively different instead of posting new insights onto Less Wrong. I wonder if my current research at Mila is a golden opportunity to do so. Mila is less connected to the Bay Area, which is a cultural hub of AI safety. Perhaps this can be the start of a fresh initialization.

One aesthetic (out of many) might be creating an AI house near Mila, in the zeitgeist of the Bay Area, where co-living creates a serendipitous environment that gives rise to friendships, collaborations, and companies. The aesthetic could maximize KL divergence with the rationalists and post-rationalists-- perhaps, instead, catering to web 3, Wall Street, or government officials. This aesthetic of AI safety could have more mainstream connotations of new finance and international relations, which could be powerful.

Honestly, the design space for AI safety aesthetics that are not correlated with the rationalists is huge. I’d like to search more intensively first. Mila could be an ideal place to craft a cohesive new value system-- the air feels “cleaner” (both metaphorically and literally) in comparison to the Bay. There’s less space taken up by established institutions, cultures, narratives, and ways of being. Perhaps this arises from Mila’s international nature. There is no singular culture, no singular way of doing things, and no singular way of thinking about AI.

I have been quietly nursing thoughts on AI safety, but the existing community feels kind of established. There is simultaneous pressure to read everything they’ve written and also a desire read none of it and start innovating from first principles with a less-correlated group of people (a fresh initialization). Even if doing this means rederiving their conclusions, it would be under a different cultural aesthetic.

I have a small hunch that creating human-like general intelligence will have built-in solutions to many original AI safety concerns instead of giving rise to them. The hypothesis here is that AI safety is already built into human-like general intelligence: I have an intuition that Bostrom’s orthogonality thesis is not even close to the full picture.

Overall craving fresh air

Like many of my colleagues, machine learning vocabulary has seeped into my thinking, including terms like rewards, value function, gradient step, constrained optimization, and search space.

Machine learning makes my thoughts clearer and better-factored, but the trade-off is losing literary flexibility, in which each word has a thousand iridescent connotations. I’ve been reading loads of Ken Liu to offset this effect-- let me know of other elegant writers to tame academic language. I want to rewrite my machine learning vocabulary with more poetic synonyms.

I am overall craving freshness.

Mathematical thinking

A well-factored mind

I want a well-factored mind. The classes that I am taking now are giving me the primitives to represent my mind back to itself in a clean way for the rest of my life. Reinforcement Learning & Optimal Control is a class full of difficult mathematical analogies: representing and re-representing reinforcement learning with stochastic optimization, linear programming, differential equations and dynamical systems, and deep learning methods.

Between Reinforcement Learning & Optimal Control and Mathematical Tools for Computer Science, I feel like somebody is taking a chisel to my mind and sculpting it into a new form. Learning how to do math feels a lot like learning how to write, which was my first love. I was fascinated how you can represent the same thought with new shades of meanings with slight tweaks in wording and syntax. In math, too, you can represent the same concept but change the emphasis or implication.

I’ll write more about this after gaining more mastery over the material. I want doing mathematics to feel like writing poetry.

Math as self-help

Some mathematician friends have commented that their mathematical thought patterns generalize to other parts of their lives. For example, one friend notices that the same thought patterns for thinking about combinatorics get used when planning social events. Which combinations of people work together for this party, and which ones don’t? Who should sit next to whom? What is the probability that two people will sit next to each other who don’t like each other?

Another friend mentions that the same thought patterns that he uses to solve proofs become active when he thinks about personal problems, like issues in friendships or relationships. Mathematical tools like deduction, induction, and proof by contradiction can be used for personal domains. What would happen if this person and I were isolated on an island (a smaller example)? What trait is common among examples? What would happen if this person did not have Y personality trait?

The more I practice proofs, the more deliberate I become about thinking about other aspects of my life, instead of representing these areas as an intuitive tangle. I want to rebrand mathematics as a general thinking toolset, instead of something niche. Mathematical people tend to already know this. But the people who are intimidated away from mathematics in early life do not get the savor these cognitive delights.

Frontier research & first principles

Graduate school so far has been simultaneously exploring the frontier of a field while solidifying first principles, reviewing again concepts that I learned many years ago. My time is split between implementing papers, running new experiments, and going over extremely basic concepts in probability and reinforcement learning to ensure that they’re baked into the structure of my thinking.

If you don’t do the problems, it’s easy to trick yourself into thinking that you understand the material. When I don’t understand the problem, I feel like GPT-3: my understanding maps onto a verbal representation but does not necessarily generalize to other representations (like spatial, casual, or modular representations).

Verbal memorization is cheap, low-dimensional, and “stupid” pattern recognition. You can easily identify GPT-3s when they talk about your specialty: they’re well-meaning people who approach you at a conference and echo back keywords about your research, but do not have a causal model of how these words link together. At many points, I am this person, so this comment is not intended to shame anyone.

To avoid “stupid” pattern recognition, I am obsessive about basics. I have a Jupyter notebook where I run simple experiments on 10-armed bandit problems: making small changes to algorithms that select the best normal distribution out of ten distributions with different means and variances. There’s always some little tweak you can throw in that changes the problem: making the distributions move, messing with the epsilon, or changing the initialization parameters.

Coding bandit problems is brutally slow (and basic), but extremely rewarding, because I understand every part of the system that I am creating.

What are “mathematical thought patterns”? Do they differ from the thought patterns of founding a company?

The thought patterns of mathematics and coding are somewhat orthogonal to the patterns of socializing and founding a company.

With the latter, you often take many actions quickly until something works. Your thinking is lateral and large-scale. The same does not apply to mathematics, which feels like rewiring your neurons to represent new primitives. Instead of seeking out some high-level pattern through induction, which is useful in super-connecting friends, networking with investors, synthesizing a literature review, or thinking about English literature, you have to map your mind onto each primitive individually, and then build the primitives up like the bricks of a house.

The experiential difference between thinking with pattern recognition, which is so useful for socializing, versus thinking with symbolic logic, which is useful for solving proofs, harkens back to ancient AI questions of sub-symbolic versus symbolic AI.

This leads to a deeper question: How is my brain representing logic? Representing sub-symbolic inputs, like intuitions, is actually easier to understand because this is the function of a neural net: my intuitions are a distributed, entangled representation of the correlations in my environment.

But what happens when the brain starts recombining primitives, like mathematical symbols, in a modular manner? What is the biological substrate of symbols? And when does a sub-symbolic representation become a discrete symbol? Experientially, this is the moment when after staring at your textbook for many hours, you finally grok the mathematical concept, and can fluidly combine the symbol with other symbols.

Long-term vision

Long-term, I am interested in Metaphysics Tycoon companies, which push at the nature of reality and lead to good outcomes for the complexity of information (not necessarily sentience, but that is a story for another day).

Serial entrepreneurship develops meta-skills: the feedback loops tend to be faster when founding topical companies (like web 3) than founding a long-term research bet.

It’s possible that the company I am working on now could roadmap its way into a metaphysical goal, like human-level general intelligence, whether directly or indirectly. If not this company, then the next one.

Mirror.xyz is still in its early days so I don’t think you can comment. Instead shout out to me on Twitter @soniajoseph_. If you liked this post, then RT the thread so others can read it too.