∵ Mari-chan ∴ 2023-09-14 ∞ 5'
I've joined a research cooperative called RADAR as an "experimentalist."
RADAR is a collective of non-traditional researchers who are looking into ways of democratizing research. It's a diverse mix of people who want to produce content that is aesthetic, informative, and that expresses diverse and creative opinions. My work as an experimentalist entails developing software experiments that enable and empower the work of my fellow researchers.
The theme for this round is "Centaur Future," which is defined as the intersection of man and machine, a forecast of what our society might turn into as AIs become more and more prevalent in our lives.
Currently I'm looking into three different little research ideas:
I argue that AI should be controlled either because AI is a driving force for Zerstörung or because AIs will eventually evolve into an existential menace to human kind. I'm not even going as far as to say that AGI is a certain development; I believe it's possible to do a lot of harm with AI even without it achieving this level of intelligence.
In order to control the AI we need to understand it, and so far, even though we have been making steady progress[1][2] in the area of interpretability, we have a long way to go.
There are many different fronts to explore in the efforts to understand AI. The one I'd like to focus on is working towards simple and understandable models.
For that my favorite example is Neural Circuit Policies (NCPs)[3]. The really cool thing about this approach is that is based on a worm's brain[4]! It's the first time we uncovered the entire brain of a somewhat complex animal and we've learned a lot! It spawned the area of NCP, which, in opposition to the current more widely used Transformers model, is an extremely simple network that contains very few nodes. It can be loaded onto small chips and achieve similar results to extremely complex Neural Network implementations[5].
Having these ideas in mind, what are the kinds of experiments we can run using these technologies or ideas? Can this be used as material to draw metaphorical questions about our place in technology and the natural world? In an evolutionary perspective, the fact that we're learning so much from such a simple animal is very humbling. What about the influence that this has in art? Can we publicize this idea in palatable way that will awake people's interest?
Cooperation was the key to our human dominance over other species[6]; positive cooperation is also, arguably, one of the most difficult issues that human kind ever faced[7]. It's pivotal for us to understand how to cooperate with AI and to enable AI to cooperate and work with each other, as well as with us[8][9].
I've recently developed the base of a system that allows LLMs to coordinate with each other[10], but I haven't added cooperative capabilities to it yet. There are also safety concerns towards open-sourcing this kind of content. Is this something we should discuss in our report?
There are a few open source libraries[11][12] that gives us tool to build cooperation scenarios using Reinforcement Learning (RL).
What kinds of projects can we develop that facilitates or improve these axis of coordination?
One of the most fascinating phenomena I've seen in software development is that of emergence. Emergence of complex patterns is observed in several areas other than software, of course—like fractals in math, markets in economics, and evolution in biology. It's mesmerizing to me to see complexity emerging from simple rules.
I've never developed my own emergent algorithm, though I've been thinking about this idea for a long time. I would love to tackle this; however, it's hard for me to imagine scenarios where we could create interesting models that could give the public a taste of what AI might become in the following years. I'm extremely cautious when drawing conclusions from models of social behaviors due to the many failures we've had in history[13][14].
One way to go is to find a theory we might be interested in and build a simplified model that tries to reproduce it. We might still draw interesting results even if our simulation is far from reality.
Here are some links to interesting simulation projects I've seen[15][16][17].
[1] https://far.ai/publication/conmy2023acdc/
[2] https://www.anthropic.com/research
[3] https://arxiv.org/abs/1803.08554
[4] http://www.wormbook.org/chapters/www_celegansintro/celegansintro.html
[5] https://www.csail.mit.edu/news/new-deep-learning-models-require-fewer-neurons
[6] https://onlinelibrary.wiley.com/doi/10.1002/9780470015902.a0021910
[7] https://slatestarcodex.com/2014/07/30/meditations-on-moloch/
[8] https://www.notion.so/marimeireles/AI-and-coordination-15814d5b41874928b08506a55c92fc61?pvs=4#d305938ccc0a416dabb88072274aba42
[9] https://www.deepmind.com/publications/open-problems-in-cooperative-ai
[10] https://techforgoodresearch.substack.com/p/yoasi-a-generative-llm-based-universe
[11] https://www.deepmind.com/open-source/melting-pot
[12] https://github.com/Farama-Foundation/Gymnasium
[13] https://en.wikipedia.org/wiki/World3
[14] https://www.investopedia.com/terms/l/longtermcapital.asp
[15] https://en.wikipedia.org/wiki/Sugarscape
[16] https://www.youtube.com/watch?v=N3tRFayqVtk
[17] https://www.youtube.com/watch?v=r_It_X7v-1E