Home
Key Moments
Projects
Books
Links
People
Blog
@johannes_hage

Links

This link section is inspired by the ones from my favourite bloggers such as gwern, guzey or nintil (José). It presents my most interesting reads of the last few months.

July 2022

  • The effective altruist work ethic and the spirit of utilitarianism
    https://www.dwarkeshpatel.com/p/ea-bi...

  • The Track Record of Futurists Seems ... Fine
    https://www.cold-takes.com/the-track-...

  • Balaji’s new book the Network State:
    https://thenetworkstate.com/

    • June 2022

  • Gwern’s GPT-3 2nd Anniversary predictions:
    https://www.reddit.com/r/mlscaling/co...

    • April 2022

  • DeepMind releases new scaling laws that contradict with the ones from OpenAI:
    https://arxiv.org/abs/2203.15556

  • PaLM - 540B parameter model by Google AI:
    https://ai.googleblog.com/2022/04/pat...

  • DALL-E-2 by OpenAI:
    https://openai.com/dall-e-2/

  • Second order effects of the rise of large language models:
    https://twitter.com/russelljkaplan/st...

    • March 2022

  • Super realistic AI takeoff scenario by gwern, based on current models and scaling effects:
    https://www.lesswrong.com/posts/a5e9a...

  • Examples of barbell strategies for everyday life:
    https://dwarkeshpatel.com/barbell-str...

  • Deep Neural Nets - 33 years ago and 33 years from now → Andrej Karpathy reimplemented one of the first neural net papers by LeCun from 1989 and analysed if we made any fundamental progress:
    https://karpathy.github.io/2022/03/14...

  • Directory of all Large Language Models:
    https://docs.google.com/spreadsheets/...

  • BigScience published several interesting blog posts this month on how they are training their 176B parameter language model:
    https://bigscience.huggingface.co/blog

    • February 2022

  • Slate Star Codex analysis of AGI timelines:
    https://astralcodexten.substack.com/p...

  • Thesis on Sleep by Alexey Guzey:
    https://guzey.com/theses-on-sleep/

  • Why Tyler Cowen with Emergent Ventures has been so successful in curating talent and why the first batch of YC and the Thiel Fellowship were so successful:
    https://www.highmodernism.com/blog/ta...

    • January 2022

  • Motivation for the roaring 20s. We choose to solve problems like alignment (+aging) not because they are easy but because they are hard!
    https://www.lesswrong.com/posts/Bseax...

    • December 2021

  • Newsletter by Sonia Joseph:
    https://mirror.xyz/soniajoseph.eth/As...

  • Sequence to understand the relationship between Progress Studies and Effective Altruism:
    https://www.highmodernism.com/sequence

  • New AGI Workshop at Mila Quebec with a bunch of videos:
    https://sites.google.com/mila.quebec/...

  • WebGPT: Improving the factual accuracyof language models through web browsing:
    https://openai.com/blog/improving-fac...

  • Gopher: DeepMinds 280B parameter model with new SOTAs across the board:
    https://deepmind.com/blog/article/lan...

  • First published work by Aleph Alpha:
    https://arxiv.org/pdf/2112.05253.pdf

  • New developments on nuclear reactors in Wyoming:
    https://www.terrapower.com/natrium-de...

    • October 2021

  • Deep Learning Diminishing Returns → Important piece but I don't agree with a lot of stuff in this. Sounds more like a propaganda piece for symbolic reasoning:
    https://spectrum.ieee.org/deep-learni...

  • Whole Brain Emulation → No Progress on C. elgans After 10 Years:
    https://www.lesswrong.com/posts/mHqQx...

  • Also check out the Reddit discussion about the WBE topic:
    https://www.reddit.com/r/slatestarcod...

  • For fellow ML Engineers - How to Train Really Large Models on Many GPUs?
    https://lilianweng.github.io/lil-log/...

  • Your chance to invest in a longevity company that went trough YC with a reasonable valuation:
    https://wefunder.com/gerostate.alpha

  • How to Train Large Deep Learning Models as a Startup:
    https://www.assemblyai.com/blog/how-t...

  • The Vitalik Buterin Fellowships in AI Existential Safety:
    https://grants.futureoflife.org/

  • 530B parameter language model by Microsoft + NVIDIA:
    https://developer.nvidia.com/blog/usi...

  • Also check out this awesome thread about the model:
    https://twitter.com/BlancheMinerva/st...

  • State of AI Report 2021:
    https://www.stateof.ai/

  • Bryan Johnson measuring all his 70+ organs to maximally reverse the quantified biological age of each:
    https://blueprint.bryanjohnson.co/

  • Awesome video about the Scaling Laws by Connor Leahy:
    https://www.youtube.com/watch?v=StLtM...

  • Just Ask for Generalization by Eric Jang:
    https://evjang.com/2021/10/23/general...

    • September 2021

  • Summary of Sam Altman Q&A on AGI predictions and GPT-4:
    https://www.lesswrong.com/posts/aihzt...

  • In What Sense is Matter 'Programmable'? → A lot of interesting ideas inspired by David Deutsch ideas in the Beginning of Infinity:
    https://jaredtumiel.github.io/blog/20...

  • Founder of NeuraLink is building a new company called Science:
    https://maxhodak.com/nonfiction/2021/...

    • August 2021

  • New chip cluster that will make 120 trillion parameter models possible (almost 100x from GPT-3):
    https://www.wired.com/story/cerebras-...

  • Scott Alexander on AGI risks:
    https://astralcodexten.substack.com/p...

  • + his answers to the comments are super interesting:
    https://astralcodexten.substack.com/p...

  • We need founder-led Biotech companies:
    https://www.pillar.vc/news/the-future...

  • If Einstein Had The Internet: An Interview With Balaji Srinivasan:
    https://sotonye.substack.com/p/if-ein...

    • July 2021

  • Sci-Hub is fundraising! (Please don't use the site to download scientific papers for free!11!1!!):
    https://sci-hub.do/donate

  • Already 2 years old, but was recommended to me recently. The Bitter Lesson. "The biggest lesson that can be read from 70 years of AI research is that general methods that leverage computation are ultimately the most effective, and by a large margin."
    http://incompleteideas.net/IncIdeas/B...

  • Also 2 years old, but must read on the arguments for a slow AGI takeoff:
    https://sideways-view.com/2018/02/24/...

  • Prompt design of neural networks (learning how to talk to an AI) will be a superpower in the future. Adding "dramatic atmospheric ultra high definition free desktop wallpaper" to the prompt for CLIP produces much more realistic images:
    https://ai-weirdness.ghost.io/the-art...

  • There will be a dope Ethereum documentary:
    https://ethereumfilm.mirror.xyz/3SV8g...

  • Funniest AI blog post I've ever read:
    https://blog.eleuther.ai/year-one/

  • How to apply for an insane amount of free TPUs as an ML Engineer:
    https://blog.gpt4.org/jaxtpu

  • Building Europes AGI:
    https://www.aleph-alpha.de/

    • June 2021

  • Research initiative that wants to go to Alpha Centauri by 2060 via an ultra-light uncrewed space flight at 20% of the speed of light:
    https://breakthroughinitiatives.org/

  • Inexperienced engineers tend to undervalue simplicity → a justification of the cross entropy loss:
    https://jacobjackson.com/cross-entrop...


    • To receive monthly updates on my most interesting reads and blog posts I'm writing, enter your email here: