I'm Eric

We report Evo, a genomic foundation model that enables prediction and generation tasks from the molecular to genome scale. Using an architecture based on advances in deep signal processing, we scale Evo to 7 billion parameters with a context length of 131 kilobases (kb) at single-nucleotide, byte resolution. Trained on whole prokaryotic genomes, Evo can generalize across the three fundamental modalities of the central dogma of molecular biology to perform zero-shot function prediction that is competitive with, or outperforms, leading domain-specific language models. Evo also excels at multi-element generation tasks, which we demonstrate by generating synthetic CRISPR-Cas molecular complexes and entire transposable systems for the first time. Using information learned over whole genomes, Evo can also predict gene essentiality at nucleotide resolution and can generate coding-rich sequences up to 650 kb in length, orders of magnitude longer than previous methods. Advances in multi-modal and multi-scale learning with Evo provides a promising path toward improving our understanding and control of biology across multiple levels of complexity.

• DNA browser playground

• blog

• paper

• github

• checkpoints

• pip install

HyenaDNA is a long-range genomic foundation model pretrained on the human genome with context lengths of up to 1 million tokens at single nucleotide resolution. It uses a simple stack of Hyena operators - a new layer based on implicit convolutions that's been shown to match attention in quality in natural language with lower time complexity. HyenaDNA enables sequences 500x longer and trains 160x faster than previous Transformer-based genomic models. HyenaDNA achieves state-of-the-art on 23 (of 28) genomic downstream tasks, and explores the first use of in-context learning in genomics. Our entire codebase is public! Go biology :)

• blog

• YouTube talk

• arxiv

• colab

• github

• checkpoints

Hyena is an attention-replacement operator that can in-context learn on sequences 10x longer, is up to 100x faster than optimized attention, has lower time complexity, by using long convolutions & gating. Hyena is a convolutional operator for large language models that can match attention quality, while scaling subquadratically in sequence length, allowing us to train 100x faster at 64k tokens, and train on sequences up to 131k tokens long. Just like attention, Hyena can be used in Vision Transformer - matching attention on ImageNet-1k, demonstrating potential for Hyena as a general deep learning operator.

• arxiv

• code

• blog

The visual world is made up of multidimensional and naturally continuous-signals, but SotA computer vision models, e.g., Transformers and CNNs, use discrete pixel representations. We present S4ND, a new deep learning layer for computer vision models, that learns continuous-signal representations of images and videos by extending work on S4 (Gu. et al) into multiple dimensions. We show S4ND can boost or maintain performance of canonical vision architectures by replacing standard 2D/3D convolutions and self-attention with a continuous and global convolutional kernel.

• arxiv

• Code

• Blog coming soon!

Our work to help fight against fake news was accepted at ICCV 2021! We created an algorithm that creates a unique image fingerprint that can tell whether an image has been manipulated (Photoshopped), which we call OSCAR-Net. Given any image in the wild, the algorithm creates a scene graph and uses graph neural networks and transformer encoders to learn an embedding of object visual features and their spatial relationships. We reached state-of-the-art of the PSBattles dataset.

• Arxiv Paper

• Project Website

Working with the Partnership in AI-assisted care in Fei-Fei Li's lab, we talked to clinicians to identify opportunities for AI to help patients in high-impact settings. Performing chest compression during CPR can vary widely in quality, leading to unneccary death. We trained a vision model to detect and rate the quality of chest compressions in order to provide real-time feedback. Our model can predict the rate of compression and detect when too much time has passed in between compressions, both key clinical metrics correlated to survival rates. We collected and annotated raw CPR videos from YouTube, and used self-supervision to increase the data signal, in particular for measuring the rate of compression.

I rotated with Leo Guibas' Geometric Computing Lab, and worked on the multi-camera perception system for an exoskeleton suit to help people walk and avoid obstacles. I created a dataset for egocentric walkable floor space detection using sparse footprint signals. Here's a sample of the depth video collected.

• Github repo

I researched human stool classification using deep metric learning at Cornell Tech. You can read about our methods in the paper here. My team crowdsourced a dataset and had three physicians annotate images of human stool. We then tried several architectures and techniques in deep learning, and ultimately we were able to predict a key clinical metric, the Bristol Stool Scale, with near-doctor level accuracy.

• Download Publication

• Seismically-resistant Bridge Columns

I worked on a research collaboration between UC Berkeley and the Tokyo Institute of Technology researching seismically-resistant bridge columns. I built a finite-element model to predict the stress-strain relation and failure modes of a novel interlocking steel reinforcement column. My work was published in the 2014 Berkeley McNair Scholar Journal.

• Download Publication

Auggi is a startup out of Cornell that's passionate about gut health. I worked on the technology that allows you to take a picture of your stool on your phone and automatically characterize and extract clinical data from that image.

TVision is a startup in NYC disrupting the TV analytics industry. I worked on a prototype to analyze how people watched TV in their living rooms as part of opt-in home studies. I used face detection and head pose algorithms to predict a person's level of attention second by second while watching shows or ads.

Here's one of the prototypes I built for TVision in the summer of 2018.

• TVision homepage

With AirCam, you can ask "hey Google, where is the T.V. remote?" and it will verbally tell you where it is. AirCam is a system that gives smart speakers "eyes" using an overhead camera. On voice command, it will turn on a camera and find everyday objects you lose around your living room, including your phone and keys.

AirCam was an applied research project with Professor Serge Belongie at Cornell and demoed at Cornell Tech's Open studio in December 2018.

• Github Code

DrowsyCam makes driving safer by monitoring your sleepiness level. When your eyes are closed for too long, it will send an alert message to wake you up. It uses key point feature detection around your eyes and calculates a ratio between the height and width, distiguishing between open and closed.

Our team of 3 engineers at Cornell built a standalone IoT device that runs on a Raspberry Pi and was demoed in April 2018.

• Github code