Your Jenkins queue looks like a city at rush hour. Jobs crawl. The master groans. Someone suggests buying bigger servers. Another person whispers about containers. Jenkins 2 just landed with Pipeline and everyone is excited to script the whole thing. Cool. Still, none of that matters if your agents do not scale. If your builds sit in line while your coffee gets cold, you do not have continuous anything. You have waiting. Let us fix that with a simple way to think about scaling Jenkins agents that works whether you are on bare metal, cloud, Docker, or that one old box under someone’s desk.
Pick your agent model before you pick your plugin
First decide what kind of worker pool you want. There are two classic models. Static agents are always on, usually long lived VMs or machines. Elastic agents appear when the queue grows and vanish when idle. Both can be great. The right choice depends on your builds and how often they run.
If your stack needs special tools or licenses like iOS signing, Android SDKs, or GPU drivers, static agents can keep that setup stable. Label them well. Use labels to route jobs to the right place and keep the master from guessing. If your builds are mostly Linux with common tools, elastic is your friend. Jenkins has strong options today. The EC2 plugin can spin agents on demand. The Docker plugin gives you clean containers per build. The Kubernetes plugin schedules pods as agents. Mesos works too if that is your world.
Pipeline makes this easier. Treat the agent as a disposable workspace and keep state out of it. Cache what you must in external stores like artifact repos and package mirrors. The less your agent knows, the faster you can grow the farm without drama.
Right size your capacity with numbers not vibes
Before buying anything, get a feel for your load. Look at queue time, not just build time. If a build runs for five minutes but waits for ten, you have a capacity problem. Track median and p95 wait across labels. You will find that one label is always jammed while others nap.
Decide how many executors per node you want. One executor per CPU core sounds neat, but IO and memory can make that painful. Start with one executor, then raise to two on larger boxes if builds are light on CPU and heavy on network or cache. Watch steal time and swaps. If you see disk thrash, cut executors. If CPU is low and wait is high, add executors or more agents.
Size agents for the heaviest common job not the biggest outlier. Move odd jobs into their own pool with a label. For example, keep a small set of big memory agents for integration tests. Keep a wide pool of small workers for unit tests. This split cuts cost and improves throughput.
Provision fast and kill faster
The win with elastic agents comes from cold start speed. If it takes five minutes to spin an agent, your queue will spike. Trim boot time with small AMIs or images, warm package mirrors, and a lightweight tool install. Baked images beat long bootstrap scripts. With Docker or Kubernetes, keep your images small and use a shared cache for Maven or npm to avoid pulling the internet every time.
Set idle timeouts so agents go away when not in use. On cloud, match your autoscaling to work hours and traffic patterns. On containers, prefer one build per pod or container to avoid dirty workspaces. Keep secrets in the Credentials plugin and inject at run time. Do not bake secrets into images. For Windows or Mac builds, you may mix static nodes with elastic Linux pools. That is fine. Just be clear in your labels and job routing.
Static farms versus ephemeral pools
Static agents are steady and simple. They shine when you require tricky drivers, heavy caches, or tightly controlled networks. You can log in and poke around. The downside is drift. Over months, tools diverge and strange bugs creep in. Cost is also a thing since those boxes run day and night.
Ephemeral agents are clean and cheap at scale. Every build starts on a fresh machine or container. Less drift. Easier upgrades. The catch is provisioning speed and image hygiene. If your image is bloated or your bootstrap is slow, your queue will grow. Also watch your Docker socket and privilege settings. Keep least privilege and split sensitive jobs to stricter pools.
Many teams run a blend. Keep a small core of static nodes for special work. Use a broad elastic pool for the rest. The master does not care as long as labels and capacity are sane.
Practical checklist for scaling Jenkins agents
- Define labels for each pool like linux small, linux large, windows, mac, docker. Keep them short and clear.
- Measure queue time per label. Aim for near zero during normal hours. Raise capacity where p95 wait grows.
- Start with one executor per agent. Increase only after checking CPU, memory, and disk trends.
- Use baked images with core tools preinstalled. Keep bootstrap scripts short and boring.
- Cache smart with a local Maven or npm proxy and a nearby artifact store. Avoid giant caches on the agent disk.
- Set idle timeouts so elastic agents vanish when quiet. Cloud bills love this.
- Keep workspaces clean. Use wipe settings or run builds in fresh containers to avoid flaky tests.
- Pin special jobs to special nodes. Use labels for browsers, GPUs, or signing machines.
- Secure credentials with the Credentials plugin and scoped bindings. No secrets in images or job logs.
- Watch the master. Too many agents can overwhelm an undersized master. Track CPU, heap, and GC pauses.
- Adopt Pipeline. Move job logic to Jenkinsfile so agents stay stateless and replaceable.
- Plan upgrades. Keep plugins and agent images in source control with version pins so rollbacks are easy.
Scale your agents and your queue turns from a parking lot into a green wave.