--- title: "Running Scrapers and Rank Trackers on a VPS (with Proxies)" description: "How to split the work between a VPS and proxies for scraping and rank tracking: sizing, proxy types, a systemd setup, pacing and retries that run for…" url: "https://serpproxies.com/blog/scrapers-rank-trackers-on-a-vps" --- [All articles](https://serpproxies.com/blog) ![](https://serpproxies.com/_next/image?url=https%3A%2F%2Fkamysmopiyetbdcmyvrd.supabase.co%2Fstorage%2Fv1%2Fobject%2Fpublic%2Fblog-images%2F162165b2-20e4-400d-b4fc-165850835014.jpg&w=2560&q=75) Guides # Running Scrapers and Rank Trackers on a VPS (with Proxies) How to split the work between a VPS and proxies for scraping and rank tracking: sizing, proxy types, a systemd setup, pacing and retries that run for months unattended. Priya Anand Proxy infrastructure November 6, 2026 7 min read A scraper or rank tracker running on a laptop stops when the lid closes, shares a home connection with everything else, and leaves from the IP your internet provider gave you. Moving it to a VPS fixes the first two problems. Proxies fix the third. This guide covers how to split the work between the two, how to size the server, and the setup that keeps a scheduled job running for months without babysitting. The short version: **the VPS is where your code runs, the proxy is where your requests come from.** Keep those two jobs separate and most of the hard decisions get easy. ## Why run scrapers on a VPS at all - **It never sleeps.** A rank tracker that checks 2,000 keywords every morning needs a machine that is on at 6:00 every morning. A VPS runs your cron jobs whether or not your own computer is on. - **Stable bandwidth.** A headless browser loading results pages is heavy on bandwidth and CPU. On a server it doesn't compete with video calls, and it doesn't hit a home connection's upload limits. - **Repeatable environment.** One Linux image, one Python or Node version, one browser build. When something breaks you know it was the target that changed, not your machine. - **Close to the target.** Picking a server region near where you search (or near your proxy pool's exit country) cuts the round-trip time on every request. ## Why the VPS IP alone is not enough Every VPS sits in a datacenter IP range, and every major search engine and marketplace knows which ranges those are. A handful of requests from a server IP usually go through. A rank tracker sending thousands of queries a day from one server IP gets CAPTCHA pages, empty results or, worse, quietly altered results that look real. There is a second problem: a server IP has one location. A local rank tracker needs to see Google as a searcher in Chicago, then Austin, then Leeds. That location has to come from the exit IP, which means a proxy in each place, not a server in each place. So the VPS handles scheduling, browsers, parsing and storage, and the proxy decides where each request appears to come from. ![Diagram: a VPS runs the scheduler, headless browser, parser and database, and sends each request out through a proxy pool that picks the exit IP and city](https://serpproxies.com/_next/image?url=%2Fsite%2Fblog%2Fposts%2Fscrapers-rank-trackers-on-a-vps%2Fvps-proxy-split.png&w=1240&q=75) ## Which proxy type for which job - **Rotating residential** for search results and rank tracking. Each request (or each short sticky session) gets a fresh real-device IP in the country or city you pick, which is what search engines expect a searcher to look like. - **Static ISP** for anything that logs in or keeps a session: dashboards, seller accounts, sites where a changing IP triggers a security check. One dedicated IP, ISP reputation, datacenter speed. - **Mobile** for the hardest targets, where only carrier IPs get through reliably. - **Datacenter** for easy targets: your own sites, APIs, monitoring pages that don't block servers. Most rank tracking stacks use rotating residential for the searches and nothing else. Start there and add other types only when a specific target needs them. ## Sizing the VPS What drives the size is whether you run a real browser. - **Plain HTTP requests** (requests, httpx, Go, curl): 1 to 2 vCPU and 2 GB of RAM handle a lot of work. These jobs spend most of their time waiting on the network, so concurrency is cheap. - **Headless Chromium** (Playwright, Puppeteer): budget roughly 300 to 500 MB of RAM and a good share of a CPU core per open browser page. Four vCPU and 8 GB comfortably run 6 to 10 concurrent pages. - **Heavy rendering or many parallel workers**: CPU is the limit. A high-frequency CPU plan finishes the same queue noticeably faster than more, slower cores. Storage matters less than people expect. Store parsed rankings, not raw HTML, and a year of daily checks for thousands of keywords fits in a few GB. NVMe helps if you keep a local database and write to it a lot. Pick a host with **unmetered or generous traffic**. Browsers download more than you think, even with images blocked, and per-GB server traffic charges add up on top of proxy bandwidth. ## A setup that survives months unattended ### 1\. Base system Use a current Ubuntu or Debian LTS image. Create a non-root user, turn on SSH key login, disable password login, and enable automatic security updates. If you run a browser, install it with Playwright's own installer (`npx playwright install --with-deps chromium` or `playwright install --with-deps chromium` in Python) so its system libraries match. ### 2\. Keep credentials out of the code Generate proxy credentials in your proxy dashboard (country, city, sticky or rotating) and store them as environment variables, for example in a `.env` file readable only by your user: `SERP_PROXY_HOST=HOST SERP_PROXY_PORT=PORT SERP_PROXY_USER=USER SERP_PROXY_PASS=PASS` Paste the values exactly as the dashboard generates them. Location and session settings live in the username, so editing it by hand is the most common reason a "working" proxy suddenly exits in the wrong country. ### 3\. Route only the scraper through the proxy Don't set a system-wide proxy on the VPS. Package updates, your database and your monitoring should use the server's own connection. Pass the proxy to the HTTP client or browser that needs it: `import os from playwright.sync_api import sync_playwright proxy = { "server": f"http://{os.environ['SERP_PROXY_HOST']}:{os.environ['SERP_PROXY_PORT']}", "username": os.environ["SERP_PROXY_USER"], "password": os.environ["SERP_PROXY_PASS"], } with sync_playwright() as p: browser = p.chromium.launch(proxy=proxy) page = browser.new_page() page.route("**/*.{png,jpg,jpeg,gif,webp,woff,woff2,mp4}", lambda r: r.abort()) page.goto("https://www.google.com/search?q=example&hl=en&gl=us") # parse page.content() here browser.close()` Blocking images, fonts and media cuts proxy bandwidth per results page sharply without changing the rankings you parse. ### 4\. Schedule it properly Use a systemd timer or cron. systemd is the better default: it logs every run to the journal, can restart failed jobs, and won't start a second copy while the first is still running. `# /etc/systemd/system/ranks.service [Service] Type=oneshot User=scraper WorkingDirectory=/home/scraper/ranks EnvironmentFile=/home/scraper/ranks/.env ExecStart=/home/scraper/ranks/.venv/bin/python run.py # /etc/systemd/system/ranks.timer [Timer] OnCalendar=*-*-* 06:00:00 RandomizedDelaySec=900 Persistent=true [Install] WantedBy=timers.target` `RandomizedDelaySec` spreads the start time so your traffic doesn't land at exactly the same second every day, and `Persistent=true` runs a missed job after a reboot. ### 5\. Pace, retry and verify - **Limit concurrency per target.** Ten parallel workers on one search engine is plenty for most keyword lists. Spread the run across an hour rather than firing everything in a minute. - **Retry on a new IP.** On a CAPTCHA page, a 429 or a timeout, wait a few seconds and retry with a new session rather than hammering the same one. - **Detect blocks explicitly.** Check for CAPTCHA markers and for results pages with no organic results. Log them as failures, never as "not ranking". - **Record the context.** Store the keyword, location, device, time and exit country with every rank. Two reports that disagree are usually measuring from different places. ### 6\. Watch it Have the job send a short summary when it finishes (keywords checked, failures, block rate) to email, Slack or Telegram, and an alert if it didn't finish at all. A rising block rate is the earliest sign that pacing, proxy type or target behaviour needs a change. ## Picking a VPS for this The checklist is short: a region close to your targets or proxy exits, modern CPUs, NVMe storage, unmetered traffic and root access to install your own stack. [PSB Hosting](https://psb.hosting/?utm_source=serpproxies) checks those boxes. It offers VPS in the Netherlands, Germany, Finland and the US on AMD and Intel CPUs with DDR5 and NVMe, unlimited traffic on every plan, ports up to 10 Gbit/s, and Linux, Windows or FreeBSD with Docker available preinstalled. For browser-heavy rank tracking, its HI-CPU plans on AMD Ryzen 7950X are the ones to look at, since rendering is CPU-bound. Pair the server with a residential pool that targets the countries and cities you track, and you have a rank tracker that runs every day, from the right locations, without your laptop being involved. ## FAQ ### Can I just use the VPS IP without a proxy? For your own sites and easy targets, yes. For search engines at any real volume, no: datacenter ranges are flagged quickly, and the server has only one location. ### Should the VPS and proxies be in the same country? It helps latency but isn't required. What matters for results is the proxy's exit location. Put the server where it is fast to reach your proxy gateway and the targets. ### How big a VPS do I need for 1,000 keywords a day? With plain HTTP requests, a small 2 vCPU / 2 GB server. With a headless browser and top-100 depth, plan on 4 vCPU / 8 GB so the run finishes in well under an hour. ### Do I need a static IP or rotating proxies for rank tracking? Rotating residential. Rank checks are independent queries, and a fresh IP per query (or per short sticky session) looks like many different searchers. Static ISP IPs are for logged-in sessions. ## Keep reading ![Cover art for “OS and TCP Fingerprint Spoofing for Proxies”](https://serpproxies.com/_next/image?url=https%3A%2F%2Fkamysmopiyetbdcmyvrd.supabase.co%2Fstorage%2Fv1%2Fobject%2Fpublic%2Fblog-images%2F9c0076d3-2343-463e-8558-47f0534323bd.jpg&w=2560&q=75) Guides Nov 4, 2026 12 min read ### [OS and TCP Fingerprint Spoofing for Proxies](https://serpproxies.com/blog/os-tcp-fingerprint-spoofing-proxies) OS and TCP fingerprint spoofing makes your proxy look like Windows, macOS, iOS or Android. See how it works on SerpProxies and how to test it. Priya Anand Proxy infrastructure Read More ![Cover art for “How to Find Repeated Customer Questions in Google and Telegram: Checking Search Results with SerpProxies and Handling Requests in Telegram Expert”](https://serpproxies.com/_next/image?url=https%3A%2F%2Fkamysmopiyetbdcmyvrd.supabase.co%2Fstorage%2Fv1%2Fobject%2Fpublic%2Fblog-images%2Fe1b1b483-a20c-4770-9296-ed7c52d0293b.jpg&w=2560&q=75) Guides Oct 28, 2026 15 min read ### [How to Find Repeated Customer Questions in Google and Telegram: Checking Search Results with SerpProxies and Handling Requests in Telegram Expert](https://serpproxies.com/blog/find-repeated-customer-questions-google-telegram) Collect the questions customers keep asking, check what Google shows for them in each region with SerpProxies, fix the pages behind them, and handle new requests in Telegram with Telegram Expert. Telegram Expert Team Guest contributor Read More ![Cover art for “Using SerpProxies with CaptchaAI in Playwright and Scrapy”](https://serpproxies.com/_next/image?url=https%3A%2F%2Fkamysmopiyetbdcmyvrd.supabase.co%2Fstorage%2Fv1%2Fobject%2Fpublic%2Fblog-images%2F74eac5a7-5f84-4c24-90f7-6eb6bb82f830.jpg&w=2560&q=75) Guides Sep 20, 2026 10 min read ### [Using SerpProxies with CaptchaAI in Playwright and Scrapy](https://serpproxies.com/blog/serpproxies-captchaai-playwright-scrapy) How to combine SerpProxies rotating and sticky sessions with CaptchaAI for reCAPTCHA v2 handling in authorized Playwright and Scrapy workflows. CaptchaAI Team Guest contributor Read More Ready to start? ## SerpProxies is for teams that move. Make your move. Credentials the moment you sign up, a free trial on residential, and billing by what you use. No contracts. [Get Started](https://app.serpproxies.com/sign-up) [Claim Free Trial](https://app.serpproxies.com/sign-up)