Following are some papers, blog posts, and videos that are generally focused in the direction of topics in research that I currently find interesting. I’m writing this to be able to share with colleges and friends as well as a sort of log to look back on in the future.
- Jack Morris Latent Space Podcast Interview
- They discuss “a new theory of information” following the V-Information paper which introduces a theory of usable information under computational constraints. The idea being that we want to measure not only the information content of a source in general, but what is useful or readable to use or any type of reader. The example Jack talks about is two documents, one in plain text and one encrypted, have the same information content, but one is readable and actually useful and the other is just text garbage.
- X-post from Jack about this “new type of information theory”: https://x.com/jxmnop/status/1904238408899101014
- They also discuss The Platonic Representation Hypothesis briefly which introduces something like: all models are trained on data from the same world and we only have one world, so theoretically, as we scale models up, they should all converge tho the same representation of the world basically.
- How much do language models memorize? - Jack Morris
- Basically experiments on estimating how much of what an LLM is trained on is perfectly memorized in the model weights in terms of bits/parameter. They reach a conclusion of ~3.6 bits per parameter (with a parameter being a weight value in 32-bit floating point representation).
- A lecture by Jack Morris on continual learning
- We need some sort of a better way to select what is “memorized” by models. In LLMs for example like in this video jack morris gives an example similar to the answer of “What is the population of the second smallest city of Albania?” should not be stored in the models weights. This is similar to what Karpathy said a while back about selective intelligence, ie. data modality (in this case, text/language) and reasoning should be learned with some basic knowledge, but everything past that should be fine tuned at inference time or temporarily inserted via RAG or something similar into the models context window.
- The Platonic Representation Hypothesis
- Here is an interview with one of the main authors walking through his thought process and ideas regarding the paper and his research direction.
- Some thoughts and notes I had while listening to it:
- In physics you want to find the unified framework to combine all theories and the platonic representation hypothesis is basically an idea of trying to go in this direction, but for deep learning and machine learning in general.
- There wont be convergence on the space of creative things like creative writing, engineering, building things etc. but there will be convergence on actual truth values like physical things and factually correct statements.
- This general idea of representation learning and the idea of things/ideas/topics grouping together in the similar representation/object groups plays a bigger role beyond deep learning. This could be a more general and philosophical hypothesis for things. Like the idea brought up in the video: we humans on earth have cats and name them cats, but what if martians had cats, in the exact same way, they would act similar around them, but just give them different names and that’s it, because they are the same thing.
- The Information Bottleneck Method
- Obviously having done my Bachelor’s Thesis on this topic, this is another thing that interests me a lot. It’s an information theoretic technique used to find the optimal balance between compression and accuracy (at least in the context of machine learning). Wikipedia say “the best trade off between accuracy and complexity.”
- In this, I’m also interested in mutual information estimators and understanding compression.
- An idea / way to think about computers that I like very much is that compression == computation. I think about the topic of an optimal computer a lot actually. By this theory, an optimal computer is also an optimal compressor and what thing in this universe can compress information the best? Black holes. Now what does “the best” mean of course, this leads as back to a theory of usable information, because the information becomes useless to use forever once it enters a black hole (at least for now). Also see this talking about black holes as computers.
- Geometric and Information Compression of Representations in Deep Learning
- I liked this because I was able to follow it pretty well, was structured similarly to my Bachelor’s Thesis paper/research, and focuses on a similar topic (not full related, but does the same thing of comparing two metrics and their trade off).
- Goes into this other topic of “geometric machine learning” that I’m becoming increasingly interested. Basically trying to understanding training, latent representations, and compression via geometry.
- In this paper, the authors investigate whether low mutual information between inputs and representations implies geometrically compressed latent spaces.
- Learning to Reason in 13 Parameters - Jack Morris
- More work by Jack Morris lol, I really like his research direction/questions.
- They basically fine tune LLMs using LoRA to increase reasoning capabilities on math and programming benchmarks using only 13 parameters! This gives some intuition as to how small the differences are now between models that are good at one thing and less good at another. A cool way to think about this is that you could basically brute force a random set of 13 different parameters and it wouldn’t be that difficult that largely increase performance on a certain task then.
- Some more that I haven’t looked to much into, but have found interesting on the surface level and want to dive into:
- Some things I would be interested in working on:
- Better optimizer’s (similar to Thinking Machines’ Modular Manifolds). Think more in terms of optimizing for compression and compressed representations?
- Mutual information estimators? Idk, don’t know to much about it yet other than the variational information bottleneck
- Look into toke ionization and improving their compression rates or something before training LLMs. See something like this, this, this, and this.