Smarter, Not Bigger: How Perforation Lets ResNet-18 Perform Like ResNet-34

This biologically-inspired technique delivers dramatically better accuracy with almost no extra compute Supercharging Ol’ Reliable If you care about vision models, you know ResNet-18 already. Perforated AI has used our dendritic optimization technology to ‘perforate’ ResNet-18, and our upgraded version is on par with ResNet-34. What does that really mean, and why should you care? My team and I think the future of AI lies in models that are smarter and smaller, and what better way to demonstrate that than by upgrading a model the industry is already intimately familiar with? ResNet-18 Still Gets Nearly 50,000 Downloads a Month: Why? In a world of Vision Transformers, DINOv2, and diffusion backbones, it might seem like ResNet-18, a model originally published in 2015, would be fading into obscurity. It’s not. ResNet-18 remains one of the most actively used pretrained vision models on HuggingFace because it hits a sweet spot that newer, bigger models simply can’t match: Speed : At only 11.7M parameters, it runs inference fast Familiarity : Virtually every ML engineer knows how to use it Transfer Learning Reliability : It provides clean, ImageNet-pretrained feature representations that transfer well to custom image datasets with limited labeled data Compute budget : ResNet-18 can be fine-tuned on an entry-level GPU; larger vision transformers can’t be For teams working in medical imaging, quality inspection, retail categorization, and similar areas where compute is a major constraint, ResNet-18 remains the right choice, not just a legacy holdover. We’re still talking about a 10+ year-old model, though, and you can’t help but wonder: What if you could make a ResNet-18-sized model that fits within tight compute constraints but performs with the accuracy of a much larger model? The answer is our perforated version of ResNet-18 , now on Hugging Face. ResNet-18 in Context Before diving into the results, it’s worth grounding ResNet-18 in the broader architecture landscape. ResNets were introduced in 2015 in the now-famous “Deep Learning for Image Recognition” paper by a team from Microsoft Research. They utilize a residual learning framework (the Res in ResNet). Without getting too bogged down in the details, it’s fair to say that this revolutionized deep learning for image recognition. ResNets won the 2015 ImageNet Large Scale Visual Recognition Challenge (ILSVRC) and became the standard backbone for a decade of computer vision research. Here’s a look at how two ResNet models of varying depth stack up against other popular computer vision models. Where each fits: MobileNetV2 — Built for on-device inference. Best when binary: it fits on the device or nothing does. Accuracy sacrifice for transfer learning is real. ResNets — The workhorse. Widely supported, well-understood, fast to fine-tune. Lower benchmark accuracy, but those numbers are on ImageNet. The fine-tuned ROI often inverts the ranking on constrained-domain datasets. EfficientNets — Strong benchmark numbers, but deceivingly slow on real GPUs. Needs extra deployment work (quantization, TensorRT) to match what the specs promise. Not ideal if you’re iterating quickly. ResNet-18 lives in a pragmatic middle ground: large enough to learn rich features, small enough to fine-tune quickly. Our perforated variant pushes it upward into ResNet-34 territory without paying the deeper ResNet-34’s steep compute costs. Our Technique: Dendritic Optimization There is a specific aspect of how modern neuroscience understands how real biological neural networks (you know, brains) work. The tree-like projections known as ‘dendrites’ on brain cells play a critical role in how the brain processes information and makes decisions. The tree-like branching structures, or ‘dendrites’, seen in red in this biological neuron, are the inspiration for our technology. Modern neuroscience has found that they play a critical role in how real brains work, a role that hasn’t been replicated in artificial neural nets until now. We’ve built an upgrade to the artificial neuron , the building block that all of modern AI is built from. Our technology adds a digital version of dendrites to the nodes of existing neural nets, making them smarter, enabling them to be smaller, and overall just better at what matters most for real-world use cases. If you want to get deep into the nitty-gritty, here is the formal paper that really explains it. The Results: How Much Better Is It, Really? The perforated version of ResNet-18 was evaluated on three standard transfer learning benchmarks: Flowers-102 , Oxford-IIIT Pets , and CIFAR-100 , canonical datasets for measuring how well a pretrained backbone transfers to new visual domains. The results of hundreds of experimental sweeps on hyperparameters tell a striking story. On Oxford-IIIT Pets and Flowers-102, the best perforated model closes 31% and 44% of the accuracy gap between the best ResNet-18 and the best ResNet-34, respectively, with only a 13% increase in parameter count when compared with the original ResNet-18. That alone would make it a compelling upgrade. But the CIFAR-100 result is the real headline: the perforated version of ResNet-18 outperforms ResNet-34 outright , while adding just 13% of the extra parameters that upgrading to ResNet-34 would require. A model with 12.5M parameters beating one with 21.3M on a fine-grained classification benchmark is exactly the kind of result that makes you rethink the bigger model = better model philosophy. On latency: testing single-image inference on an AMD Ryzen Threadripper PRO CPU across the full Flowers-102 dataset showed latency proportional to parameter count, meaning the perforated model runs only marginally slower than vanilla ResNet-18, nowhere near ResNet-34’s speed penalty. The perforated ResNet-18 model ships with a trained ImageNet classification head and works out of the box for ImageNet inference. For transfer learning to a new dataset, simply replace the final FC layer as shown in the usage section below. Does This Make ResNet-18 Obsolete? It almost feels wrong to say it, but, generally, yes . If your deployment environment is so tightly constrained that the difference between 11.2M and 12.5M parameters genuinely breaks your size or latency budget, stick with vanilla ResNet-18. For essentially every other use case: if your system can run ResNet-18, it can run this model, and it will run it better. The perforated model matches ResNet-18’s footprint closely enough that it’s not a meaningful trade-off; you’re getting a model with closer to ResNet-34-level accuracy at a ResNet-18 size, with the associated GPU speed and efficiency, for free. How to Use the Perforated ResNet-18 Installation pip install perforatedai Load the Pretrained Model import torch import torchvision from perforatedai import utils_perforatedai as UPA from perforatedai import library_perforatedai as LPA # Create base model architecture base_model = torchvision.models.get_model('resnet18', weights=None, num_classes=1000) # Convert to perforated architecture model = LPA.ResNetPAIPreFC(base_model) # Load pretrained weights from HuggingFace model = UPA.from_hf_pretrained(model, 'perforated-ai/resnet-18-perforated-cascor') Adapt for Your Model for Transfer Learning import torch.nn as nn # Replace the final FC layer for your task num_classes = 102 # Example: Flowers-102 model.fc = nn.Linear(model.fc.in_features, num_classes) From here, standard PyTorch fine-tuning applies. The architecture change is entirely encapsulated in the perforatedai library. Your training loop, optimizer, and evaluation code don’t change. You Can Add Dendrites to Your Own Architecture Do you have a model that you’d like to try dendritic optimization on? Perforated AI was recently added to the official PyTorch ecosystem , so now you can perforate your own models. The free, open-source version is easy to use and includes coding agent skills to get started. Conclusion We think the perforated version of ResNet-18 represents something worth paying attention to. Dramatically improved accuracy on small models approaching and even exceeding the performance of larger traditional models delivers real value both to the engineers building the models and the people and businesses who benefit from their performance. For practitioners in constrained-compute environments who rely on ResNet-18 for transfer learning, this is a direct upgrade worth evaluating. For researchers, it’s early evidence that dendritic optimization may deserve a place in the standard toolkit alongside residual connections, attention, and other architectural primitives we now take for granted. For end users, from hyperscalers to startups, better models equal a better bottom line. The model is live on HuggingFace. See for yourself, testing it takes an afternoon. Model: perforated-ai/resnet-18-perforated-cascor | GitHub: PerforatedAI/PerforatedAI Smarter, Not Bigger: How Perforation Lets ResNet-18 Perform Like ResNet-34 was originally published in Towards AI on Medium, where people are continuing the conversation by highlighting and responding to this story.