The Family Proxy

by Christopher Rice on 5/11/2010 2:45 AM EST
Comments Locked

96 Comments

Back to Article

  • dilidolo - Tuesday, May 11, 2010 - link

    I use pfSense as my firewall and wireless AP. Just use an old PC with pci wireless card and you are set, not even a wireless router
  • JarredWalton - Tuesday, May 11, 2010 - link

    You can of course go that route. You could make the Linux box your router and DHCP source. But most people already have a wireless router so connecting to that makes sense to me. I'd have to purchase a wireless PCI card to put in a Linux box, and traditionally wireless cards have far more limited range than routers (due to the single small antenna and sub-optimal location of being behind a large computer).

    But as with all things Linux, there are many ways to set things up. This was a short article to introduce a useful concept that many users likely haven't thought about.
  • leexgx - Tuesday, May 11, 2010 - link

    i was going to question the same thing but then i thought last time i setup IPcop i would of done the same thing disable the DHCP server and use it as an HUB/AP

    main thing i loved with IPcop was the bandwidth throttle i could cap it 5KB under my upload limit i could set utorrent or emule to full upload speeds and i could still play games online lag free (but removed due to lack of supporting uPnP i needed it for MSN remote support and games that Required Upnp {bit lame} at all hope this review used an linux distro with an upnp server on the lan side and i had an look and it did not)
  • ninjaproxy - Monday, May 20, 2013 - link

    With a proxy site you can browse your favorite web sites anonymously and even from behind a firewall with blocked ports. Whether you are on the job, at school, a college university, a public terminal or anywhere else with a web browser.

    http://www.ninjaproxy.eu
    http://www.ninjaproxy.org.uk
    http://www.vtunnel.ca
    http://www.fbproxy.us
    http://www.ninjacloak.us
    http://www.proxyninja.us
  • Zok - Tuesday, May 11, 2010 - link

    I've always wanted to tackle something like this, but the power draw has always been the biggest turn off. If average power consumpton hovers around 100W, that's about $105/year for the American average.

    I'd love to ditch my router/AP, but, sadly, I just don't see the benefit of replacing it with something that's significantly more power hungry, unable to act as a dual-band 802.11n AP (last I checked, the drivers weren't out yet for AP mode, if ever), would cost nearly as much as a good dedicated device to outfit with 3-4 additional LAN ports, and is typically physically large and hideous - not to mention the PITA of hours of initial setup and troubleshooting, when you're not a Linux expert.

    In my dreams, I wish there would simply be a beefier all-in-one WRT54G-like device running an Atom, supported dual-band 802.11n radio(s), and SATA, allowing for full-blown Linux in a compact package that would be so win.
  • JarredWalton - Tuesday, May 11, 2010 - link

    Depends on where you live, obviously, though the national average appears to be just over $0.10 per kWh:
    http://www.eia.doe.gov/electricity/epm/table5_6_a....

    That's why I mention the attractiveness of a Mini-ITX setup, particularly with Atom or similar. Most nettops use a maximum of around 25W, so that would be 1/4 the cost of a typical system, and a nettop is about the same size as a standard router. Too bad they don't have two NICs.
  • Zok - Tuesday, May 11, 2010 - link

    I've explored such a scenario. Unfortunately, at this point, the best it seems I can do is the Mini-ITX router going to the 802.11n AP. Adding another device into the network (upfront cost + power), without removing any others seems like a poor value proposition to me, at least with FiOS speeds.

    Don't get me wrong, I like the idea. I'm just waiting for someone to come along and actually produce a fully-integrated device (x86 CPU, memory, mobo, 4-5 Ethernet ports, Linux-AP supported 802.11n radios, case - without using large expansion cards/slots) that is under $250 and isn't awkwardly large and ugly. Having the 4-5 port switch and (potentially) the radios integrated into the motherboard itself is what I am looking for, although I'd be OK with Mini-PCIe for the radios.
  • taltamir - Tuesday, May 11, 2010 - link

    Don't get me wrong, I like the idea. I'm just waiting for someone to come along and actually produce a fully-integrated device (x86 CPU, memory, mobo, 4-5 Ethernet ports, Linux-AP supported 802.11n radios, case - without using large expansion cards/slots) that is under $250 and isn't awkwardly large and ugly. Having the 4-5 port switch and (potentially) the radios integrated into the motherboard itself is what I am looking for, although I'd be OK with Mini-PCIe for the radios.

    I am pretty much in the same boat.
    at my power rate a 24/7/365 device costs me about 1$ per watt per year.
    so putting a 75 watt old computer there is another 75$ a year...

    I can't wait to ditch my router for a linux based router, but it doesn't seem to be happening.
    Actually, it doesn't even need to be an x86 CPU, any CPU will do. ARM and PowerPC are both supported by linux
  • ChrisRice - Tuesday, May 11, 2010 - link

    There are various power saving techniques you can use to keep the power down as well. If your processor supports speed stepping you can utilizing that as well as powering down actual hardware until use is needed. I wanted to keep the article short so I didn't get into those areas.
  • clarkn0va - Friday, May 14, 2010 - link

    http://www.newegg.ca/Product/Product.aspx?Item=N82...

    A little over your stated budget, but good value nonetheless. Throw a supported wireless card (try ubnt.com) into the spare slot, add your favourite distro and life is good.
  • rahvin - Tuesday, May 11, 2010 - link

    Jetway motherboards can be configured with daughterboards that don't use the PCI slot. One of these daughterboards contains 4 gigabit realtek network interfaces. I'm running this on my linux firewall/router and it works beautifully. The only issue that was an eye catcher was the original 10/100 ethernet on the MB got configured as eth4 after adding the daughterboard which I didn't expect.
  • Zok - Tuesday, May 11, 2010 - link

    Wow. You're right. Jetway AD3RTLANG gives 3 x 10/100/1000. Pair that up with one of their fanless Atom board with daughert board support (NF92-270-LF or perhaps the dual-core version) and we might have a winner.
  • Zok - Tuesday, May 11, 2010 - link

    My enthusiasm got the best of me... That does sound pretty slick, but I forgot my other major gripe - 802.11n AP support (dual-band/radio, if possible). Any advancements on this?
  • rahvin - Tuesday, May 11, 2010 - link

    Use the PCI slot to add a PCI wireless card. Most of the Jetway boards come with a PCI expander that tips the PCI slot parallel to the motherboard. With the right case you just add the wireless PCI card (make sure it has FOSS drivers) and you are good to go. Or you can add a PCI card that takes a mini-pci card and then hook up an external antenna. Or you can do what I did and buy a wireless AP extender that connects via network, they are just the radio and a network interface so you just run DHCP and services over the network point and everything is automatic (although if you want security like WPA2 you have to run it on the firewall/server not the AP.
  • JarredWalton - Tuesday, May 11, 2010 - link

    So I've ordered a USB to Ethernet adapter, and when it arrives I'm going to try setting this up on a laptop. The 100Mbit USB-Ethernet will connect to the Internet (since my broadband caps out at under 20Mbit) while the onboard 100/1000Mbit (depending on laptop) will serve the home network. I'll then give this sort of setup a shot using both an Atom netbook and a CULV laptop to see if there's a noticeable performance difference (other than the netbook being limited to 100Mbit).

    As a side note, I plugged my current box into a Kill-A-Watt device this morning to see how much power it's using. The final tally: 125W! Ouch. What's really odd is that using the acpi-cpufreq package didn't help power at all. The initial setup was for performance, with the CPU at 2.40GHz all the time. Changing to the ondemand governor dropped the CPU speed to 1.6GHz, but power draw remained essentially unchanged. (It may have dropped one or two watts on average, but nothing significant.)

    All of that points to the reason I included the comment on the end about old hardware and electricity costs. I thought the box would be closer to 100W, but obviously not. A CULV or Atom netbook on the other hand will get me down to ~10W I think. :-)
  • Zok - Tuesday, May 11, 2010 - link

    I can do that now, with my current router/AP. The downside - I'm not removing any current hardware from my setup, which is my goal.

    In regards to my previous post, I was more concerned with Linux software support for 802.11n in AP mode. Last I checked, it only supported client mode. I'll do some research tonight and see if there have been any advances. It's hard to give up 300 Mbps (MIMO) for 54 802.11G or even 130 Mbps ("Plane Jane N").
  • rahvin - Thursday, May 13, 2010 - link

    Is the goal to simply eliminate devices or are you concerned about power use and flexibility? A mini-itx platform with 2.5" drives and a wireless card is going to be far more flexible than a router and it's going to use far less power than both combined. See the beauty of the home server/firewall on linux is that you can run so many services that you can't on a router. Caching DNS, Caching transparant proxy, samba, email and web filtering and AV scanning, etc. I couldn't use just a router anymore because I would pull my hair out with the limited functionality.
  • dezza - Tuesday, May 11, 2010 - link

    I know you told us you're relatively new to Linux, but I would like you to consider this:

    * A rolling release brings the newest exploits
    * Configs are not specialized for the distribution and configured to work in conjunction (Like Debian)
    * ArchLinux is primarily not a server operating system. I use it as a great workstation and the happiest I've had for years .. (Earlier running Debian, Gentoo, etc.), but I've never had any great experiences with it as a server. Most of the server-packages does not work out-the-box like on Debian.

    I would choose FreeBSD/Debian for a simple proxy.

    Also I would agree that anyone who is tempting to learn Linux starts with discovering ArchLinux and it's wiki http://wiki.archlinux.org there is everything you need to know and with a good friend by your side or a friendly IRC-channel you will be up and running quickly and will not encounter the same problems like people trying out Ubuntu, because you've already learned the hard steps by configuring it yourself. On Ubuntu people always stall on simple small problems and start bumping threads in the forums, simply because they're stuck with a default system looping around in driverproblems and Xorg configuration lines

    Ubuntu ends the same place as all other easy distributions - So you can just as well use your time to read a simple installation guide step-by-step like supplied by Gentoo and ArchLinux and learn much more in shorter time than you will use writing on the Ubuntu forums for common problems.
  • JarredWalton - Tuesday, May 11, 2010 - link

    Chris is hardly new to Linux... I'm not even "new" per se -- I used Linux (Red Hat and SuSE, plus the HP boxes at the labs) back in college in the 90s. Chris is a senior Linux engineer/admin/whatever for a major company, so he deals with configuring and running large corporate systems on a daily basis. And he likes Arch. You don't have to update regularly with a rolling release, but it allows you to do so painlessly at any time. I think the bigger reason he likes it is that you can get an Arch install lean and mean. You only install what you feel is necessary and nothing else.
  • dezza - Tuesday, May 11, 2010 - link

    No you don't have to update it, but that will leave exploits open ..

    If you update you have a new risk of newly forged exploits with the rolling release.

    There is a good reason why FreeBSD and Debian devs keep packages for a while .. I would not categorize ArchLinux as suited for servers.
  • ChrisRice - Tuesday, May 11, 2010 - link

    Freebsd would certainly be my second choice in home firewall systems "First in the corporate scene". That being said I've always been a fan of having the newer packages of Arch compared to Deb order to get many new features that you would be without in a Deb environment. As far as bugs/security holes because its a rolling compared to the bugs/security holes on a Distro with a slow moving release system, I think they both have their own downsides.
  • mfenn - Tuesday, May 11, 2010 - link

    I agree with dezza that Arch should *not* be used in a "set it and forget it" box. The great thing about Debian or Red Hat is that you can choose to only receive security updates. The maintainers also backport security fixes for the supported life of the release (which for RHEL is 7 years!). Arch only provides the upstream package versions, so if you want the latest security fixes, you also get the latest functionality-killing bugs. Also, for somebody who isn't religiously running "pacman -Syu" every week or so, Arch will quickly fall into the dist. upgrade hell that you get with other distros. You've got to realize that rolling release doesn't eliminate the dist. upgrade problem, it just allows the user to spread the problems across a longer span of time (e.g. I can update every month for a year and encounter 1 problem each time, or I can upgrade every year and encounter 12 problems). For an infrequently updated system (i.e. one build by any reader of this article because let's face it, if they were Linux geeks, they would have one already) you *will* have upgrade problems. In summary, a growing trend in the Linux community is to treat Arch as a panacea, which it most certainly is not. It's great for some things (desktops for tinkerers, development with the latest and greatest, supporting oddball hardware), but a server distro it isn't.
  • KaarlisK - Tuesday, May 11, 2010 - link

    About x1 in x16 slots:
    Could you please test that? :D You have the motherboard.

    And why is cache_memline always half the RAM? Even if you have, for example, 8GB?
  • JarredWalton - Tuesday, May 11, 2010 - link

    I think Chris assumes you don't have that much RAM. You probably only need to use half the RAM or leave 1GB free, or you can get by with just caching 512MB in RAM. I have my proxy set as 2GB RAM, and so most of the data comes across from the proxy at GbE speeds. If it goes to HDD than the speed will drop to around 50MB/s, which is still plenty fast.
  • KaarlisK - Tuesday, May 11, 2010 - link

    Thanx for the explanation!
    So basically my usage pattern can determine the cache size, and there IS use from a large cache, as your 2GB example shows.
  • ChrisRice - Tuesday, May 11, 2010 - link

    I would recommend starting with a smaller cache then tweaking up. I run a 256MB ram cache and that works just fine for me. That being said if I had more ram on the hardware I am using, I would run at least 1GB.
  • KaarlisK - Wednesday, May 12, 2010 - link

    Thanks for the reply.
    And thanks again for the article! This might finally be the way to sneak... a kind-of replacement for WSUS on a certain network. I know it's horrible, but I do not really have a choice.
  • enterco - Tuesday, May 11, 2010 - link

    Hi!

    I would like to make few observations:
    - many owners of an old PC suitable for a caching proxy are using ATX motherboards and enclosures, making the proxy a 'big noisy box', just good to keep in the basement, if you have one.
    - an old P4 computer will add enough bucks to the electricity bill
    - a typical computer user is not familiar with the requirements of configuring a Linux box, and will avoid this kind of setup.
    - the most bandwidth hungry applications in a home is not a HTTP download, but P2P transfers.

    I, personally, don't have the basement, and I don't want the noise made by such a box, neither to waste space or money in electricity in my home. So, instead of bringing back to life an old machine, I would prefer to configure QOS on a wireless router.
  • pkoi - Sunday, May 16, 2010 - link

    Go the VMware way.
  • medys - Tuesday, May 11, 2010 - link

    These days virtualisation is the answer :)

    I know that most of people do not need so much as I do and a lot of them do not care about backups, but in case you do, there is a great way to have everything you need in one box :)

    Get some semi old PC with at least 2GB of RAM (4GB is recomended).
    Install a distribution of linux on it that can run virtualisation software (VirtualBOX, vmware server, KVM).
    Configure the linux as NAS server.
    Install virtualisation software.
    Create virtual machines for anything you want :) router, proxy, LAMP, application server etc....
  • KaarlisK - Tuesday, May 11, 2010 - link

    A semi old machine probably won't have any virtualization instructions.
    I fear to think what will happen when you chain these virtual machines together :D though I may be completely wrong.
  • JarredWalton - Tuesday, May 11, 2010 - link

    Virtualization may be the answer, but what was the question? "What is the answer to life, the universe, and everything?" Virtualization! And 42.

    Good night, all!
  • ChrisRice - Tuesday, May 11, 2010 - link

    I agree with this statement completely. Setting up a KVM/VMware server is a great way to get even more use out of your centralized computer. With the proper hardware and switching you can go one further and make it fail over in case of an outage "Ya I know a bit over done for home but allot of fun".
  • mindless1 - Tuesday, May 11, 2010 - link

    Once you get your target for memory caching you can determine how old a system will suffice based on reasonably upgradable memory capacity. For example, a Pentium II/350MHz with 768MB of memory would suffice for many home users, but alas you probably want a more modern, not worn out old, hard drive that uses SATA.

    It's not hard to get power consumption down low though, follow the same standards for underclocking that you would for overclocking, remembering that the typical bottlenecks are not memory or bus frequency, or CPU processing capability. As with the hard drive there is yet another issue, a box like this you would typically plan to set up and use for years at a time so if you pick a box already 5 years old that would've lasted 10 years total, do you want to have to do the project over again for no reason other than to avoid having spent a few dollars more now?
  • chromatix - Tuesday, May 11, 2010 - link

    I've had a setup very like this for about a decade, using everything from a 486SX/25 running Red Hat 6.1 up to an Athlon-XP 2500 with a RAID-5 array, and back down to a redundant PowerBook G3 running Gentoo. I happen to run a caching DNS server as well on the same box, partly because at various times I've found ISP DNS service to be unreliable.

    The G3 is *silent*, and sufficiently powerful both to do it's job and compile Gentoo updates. It's the best use I've found for an old PowerBook ever. It even still has a few minutes of life in the built-in UPS, and if I wanted to I could extend that to about 5 hours for about €100. ;-)

    There are some downsides to Squid. As an enterprise-grade tool it has a very slow development cycle, and the stable versions do not yet support IPv6 and - as Peacekeeper demonstrates - have trouble with some recent webservers. In general though it works well.
  • EvilIgor - Tuesday, May 11, 2010 - link

    I would recommed Smoothwall instead. But this alot more powerfull then just a proxy.
  • ChrisRice - Tuesday, May 11, 2010 - link

    Smoothwall is another great product that I have used in the past. Its a very easy setup with a nice GUI interface. However I have found over the past few times I have worked with Smoothwall that it is lacking in features compared to a more traditional Linux setup.
  • rahvin - Tuesday, May 11, 2010 - link

    Chris,

    I believe you are making a mistake to assume that the user has configured their interfaces exactly as the 2-interface example. If I were you, I would edit the article and add an edit the of interfaces file to correct the zones to match the network configuration the user has chosen or tell the user that the internet needs to be on a particular interface.

    FWIW I agree with your choice of Shorewall, although less popular it's far more configurable than most of the other packages although you have to be accustomed to the "linux/unix" way of management (ie text configuration files). I'm glad you pointed users to it, although I would be happier if you suggested Debian as it's easier to manage security updates IMO, and that's a critical feature for a persistently connected box.
  • LiamC - Thursday, May 13, 2010 - link

    What features (mentioned in your article) would it be missing? If you just want a (transparent) proxy that handles account authentication and DHCP, then Smoothwall fits the bill--and also acts as a firewall. And it is very easy to setup.
  • Exodite - Tuesday, May 11, 2010 - link

    I suppose I'm spoiled by living in a country with decent network standards but to me the obvious solution would be to just get rid of the router and modem and plug your home switch straight into the ethernet wall outlet. :)

    Oh well.
  • mindless1 - Tuesday, May 11, 2010 - link

    What's the "ethernet wall outlet" supposed to be? If you mean plug your switch straight into a modem of some sort, you're better off having the extra layer of security afforded by the NAT feature on a router. That is especially true with Windows based PCs, nevermind that for many people use of a router also gives them wifi capability.
  • Dravic - Tuesday, May 11, 2010 - link

    As a previous poster mention you should look at Smoothwall express 3.0. My current incarnation is running on an old duron kt 266 platform with 1gb of ram and that is complete overkill for something like this. I would also look at the benefits vs the performance loss.

    With a few as 4 or 5 pc's you are probably getting reduced web browsing performance for the benefit of reduced broadband usage on a small amount of large files. Is it worth having a slower internet 99% of the time in order to increase download time of for 1% of the time? Remember for every image you hit you now have to query this proxy to see if the image is stored locally and possibly if an update copy of the image exist at the original source. Configuration of the proxy will be key. How much do you store , and how long do you cache items before expiration can have a massive effect on regular browsing.

    Proxies are really meant for networks with significant number of users hitting the same content repeatedly. Caching the web objects of the most frequently viewed website of 100 people provides real savings in bandwidth and increased browsing speed. For a small group of people the bandwidth saving are usually mild, but now you have increased browsing times across the board.

    I think you would better served using a qos solution (also in smoothwall 3.0) over a a squid proxy. On my fios 20/5 mb line qos overhead eats ~ 1mb of total capacity.

    In my home network (6 pc's and a few Internet appliances) neither qos or a proxy were beneficial with fios(i know not everyone has 20/5 internet, but this held true even when fios was 10/2). When i was on dial-up-upl the proxy was great for hitting multimedia heavy sights like ESPN.

    Either way i do recommend anyone with the know how build there own firewall appliance if they can stand the energy cost. The consumer grade firewall/gateways really are poor and while getting better really don't offer the range of services something like smoothwall (m0n0wall, ipcop, pfense .. etc) does.

    Other then my philosophical difference on the benefits, good article. A followup with the most widely used pre built solutions with some kind of browsing benchmarking would be a nice follow up.
  • dezza - Tuesday, May 11, 2010 - link

    I totally agree.

    Actually the thing that brought me to this site was because a friend once told me that I would not benefit anything from having a "family"-proxy .. And I would think that these comments support that conclusion.

    I would say if you're about to do this to it 100% and QoS and DHCP, etc. there is no point in having a server consuming 300-400W running JUST for a proxy that maybe even slows down browsing in the end and brings more maintenance to your home network.
  • ChrisRice - Tuesday, May 11, 2010 - link

    For the two above posts I need to get some data/graphs together to add to the article. Much of what is being said above is simply not true. I'll try to work on this over the next day or two.
  • bob4432 - Tuesday, May 11, 2010 - link

    i am not running what this article is about software wise, but my home server is a skt939 3000, 1GB ram, 60GB main drive, 500GB image holding hdd and a 120GB misc hdd running an old pci gpu and i think 3-4 80mm fans on a antec earthwatts 380W psu. my simple network setup is a asus wl520g (i think that is the model number) w/ tomato 1.27 in addition to a 8port GbE switch. the reason for explaining all this, is that combined it all pulls 60W from my ups which was verified by a kill-o-watt.
  • imaheadcase - Tuesday, May 11, 2010 - link

    I remember using WinProxy way back in the day (early 90s) for dialup. It worked EXCELLENT. But why "save" bandwidth with something like this when you have broadband?

    Its not going to save much at all.
  • ChrisRice - Tuesday, May 11, 2010 - link

    As the article refers to "Family Proxy" you could easily run out of bandwidth with broadband. For example if you have a few bandwidth hog room mates or have a wife and kids the savings are very much there. This is also the most simple setup of a proxy which could be expanded on to work with ftp and other ports. I wanted to keep it pretty simple but maybe there is interest in a more advanced setup?
  • micksh - Tuesday, May 11, 2010 - link

    How does proxy affect browsing experience? I assume there will be additional latency. Did you compare web page loading times with and without proxy?
    And how much (in seconds or minutes) does it help when you download large file second time? Does it make things faster if other PCs are doing something else on web?

    I actually tried similar setup hoping to make web browsing faster. I had Safesquid on Ubuntu on relatively fast Core 2 Duo PC using 6 Mbs AT&T DSL. It didn't help. Most web servers give content using "post" method so pages could not be cached. I enabled pre-fetching but I guess I could not configure priorities correctly. Pre-fetching made current page to load slower. Without prefetching things still seemed a bit slower because of the latency that additional box gives.
    Since I moved to 18 Mbs U-Verse and things are good without proxy.
  • spazmedia - Tuesday, May 11, 2010 - link

    Its nice to see an article on anandtech about Linux. Once you get the hang of it, most Linux distro are FAR simpler to configure then Windows as the config does not change much from distro to distro and from version to version. Also as others have pointed out smoothwall is quite easy to configure. Another useful tool for configuring all aspects of a linux box remotely through https is webmin (http://www.webmin.com/) I've tried it with Fedora and Debian/Ubuntu and it probably is a bit more functionnal with Debian. For ease of use nothing beats Suse though (from Novel)
  • spazmedia - Tuesday, May 11, 2010 - link

    BTW for those looking for power savings, its a bit more expensive but a gread idea for this application to use an Atom or low power celeron processor...
    Or the best is an old laptop (probably need to buy an extra PCMCIA NIC though). Plus you get battery backup if its not too old and battery not worn out. Having said that setting up Linux on most laptops is not trivial given custom hardware most manufacturers implement.
  • mindless1 - Tuesday, May 11, 2010 - link

    Given a motherboard with the features needed, you can underclock most system configurations to reach a power level similar enough to an Atom or Notebook... especially if the Atom system is using the relatively power hungry (for its feature set) 945 chipset.

    For example, take what many call a power hungry setup like an Athlon XP @ 60W peak load. Undervolt and underclock to 1/4th it's original speed and you still get enough performance for most people's needs, but only 15W at peak power. Now consider that it is hardly ever running at peak power, that the power savings is a single-digit # of watts at most. Only catch is, some motherboards limit how low you can undervolt, the above example only assumed power savings from running at 1/4th clock frequency but if you can undervolt too, the power savings go up even more.
  • MySchizoBuddy - Tuesday, May 11, 2010 - link

    can i use sheevaplug as my cache proxy
  • MySchizoBuddy - Tuesday, May 11, 2010 - link

    OK it can
    From SheevaPlug FAQ
    Sheevaplug can be used for
    "Web proxy, enabling fast, cached access to your favorite web sites"
  • ibloomfield - Tuesday, May 11, 2010 - link

    i use squid for the sole reason of working my way around filters at school.

    if you set up ssh to work off port 443 (in case outgoing 22 is blocked) and then tunnel squids port through the connection (port 3128) then you can set the browser to proxy 127.0.0.1 port 3128 and your are set to go.

    easy workaround
  • Pinski - Tuesday, May 11, 2010 - link

    Or you can just use ssh -D localhost:### host, and you'll setup a SOCKS proxy via SSH and use that to browse the web without ever having to deal with configuring/running Squid.
  • Pinski - Tuesday, May 11, 2010 - link

    Woops, forgot ### would be a port number of your choosing.
  • mfenn - Tuesday, May 11, 2010 - link

    Should say /etc/resolv.conf instead of /etc/resolve.conf
  • JarredWalton - Tuesday, May 11, 2010 - link

    Fixed... I blame *nix and their fetish for dropping letters. LOL.
  • bob4432 - Tuesday, May 11, 2010 - link

    " I set my maximum size to 2048MB in order to retain everything up to a CD ISO"

    am i reading this wrong? what cd iso is 2GB in size? why not set it too 700MB or 4.7GB? or even 8.5GB for a d/l dvd?
  • RamarC - Tuesday, May 11, 2010 - link

    you post a great little article! keep it up since there doesn't seem to be anything new/big/amazing on the hardware front. ;)
  • eleon - Tuesday, May 11, 2010 - link

    I really encourage everyone to use or try linux, and to reuse old hardware. but this concept is the wrong solution in so many ways.

    The main advantage of caching proxies is not to save bandwith, it is to reduce the downloaded data volume.
    If you have the problem that the bandwith of your internet connection isn't shared between your client and/or applications fair enough, you need to think about QOS not a proxy server.

    a rolling release distribution for a router???? I use archlinux myself on my laptop, and I like the rolling release cycle and the cutting edge packages on my Desktop. but it's really the wrong distribution for a infrastructure-box like a router. your argument that you never have to care about updating anymore is wrong. I would say you have to care/worry everytime you are updating! The advantage of a distribution with stable releases is that you set up the box, and if it's up and running you have only securityupdates. this means only minor updates and there are no configuration changes. with a rolling release you have major versionupdates and there is a greater chance that your config isn't working after updating a package. so there a two szenarios: you update frequently and risk everytime to break the system (which provides your internet-access). or you don't update, und your router/firewall may have serious security-issues. so using a rolling-release-distro on a router isn't a good idea at all!

    use a pc that needs more than 100W for this? maybe you should think about investing this energy-costs in a faster internetconnection?

    I was thinking about a caching proxy myself, but for a shared G3 connection which has a data volume limitation of 6GB/Month. in this area a caching proxy can make sense, and you can add something like ziproxy to reduce the transmitted data by compressing the pictures. but one youtube video produces more traffic than 100 pictures. so whats the point, and squid doesn't cache dynamic content like flashvideos.

    so for your problem/goal to have a "fast surfing experience" while your family is doing what ever on the internet, you solution is QOS, which can handle this very effectively. use embedded hardware to be energyefficent, and use a specialized router distribution ( openwrt, pfsense ,... http://en.wikipedia.org/wiki/List_of_router_or_fir... ) so that you don't spend lot's of hours to get it running, which is really inefficient too.

    but if your goal is to learn something about linux, your family proxy project is the way to go! :)
  • Dravic - Tuesday, May 11, 2010 - link

    My reply was similar to yours a qos solution is what would fit best in this situaion, unless you're dealing with usage caps or low bandwidth service. I've tried this several times over the past ~7 years at home and the browsing experience was noticeably slower when using a proxy. The extra latency of even a hashed disk look up of an object is slower then just gettng the object on a broadband connection.

    But I was told this just wasnt "true" .. well see

    On a saturated link i can see where a proxy would help because your not going over the link, but that is the job of qos. I'd like to see FULL page load metrics for both types of data retrieval (while link saturaded and unencumbered).
  • JarredWalton - Tuesday, May 11, 2010 - link

    I'm not sure some of you are on the same page as me. First, my particular setup was done purely for initial testing. As I comment (multiple times), it's complete overkill--both from a hardware performance as well as a power requirement perspective. From the conclusion:

    'Our only recommendation is that you consider the cost of electricity compared with the hardware. Sure, Linux will run fine on "free" old hardware, but a proxy server will generally need to be up and running 24/7, so you don't want to have a box sucking down 100W (or more) if you can avoid it.'

    We're not saying you need to do Arch, or you need high-end hardware. In fact I'm going to try setting up a proxy with a CULV and Atom laptop to see how that works.

    As far as QoS, we never even mentioned that. The point of a caching proxy is to avoid going out to the Internet multiple times for the same data. For me in particular, where I review lots of laptops that need frequent updates, and I have to get new video drivers regularly, the idea of a proxy means that I can speed up the process for quite a few things. I'm not worried about "saving bandwidth" in the way you're discussing, though if you had a plan that charged you for downloading over a certain amount it might be useful. I'm interested in speeding up patching and such.

    Hence, the comments about wishing Steam would work with my proxy... as it stands, I have to manually copy updated files from one PC to another, or else let each download the latest updates manually. L4D2 has had a few 200MB+ updates recently, and I'm sure I've downloaded that on various PCs/laptops at least four times. At 1.5MB/s, it can take a while, especially if I just wanted to play a quick game.

    Everything we discussed in this particular article can easily be applied to Red Hat, Debian, SuSE, Ubuntu, or whatever favorite distro you choose. As a typical non-Linux user, it amazes me how much time people spend arguing over the benefits of their chosen distribution. It's attitudes like that that frighten away potential converts more than anything. Instead of arguing about why one of our specific configurations was bad, why not point out the good?

    Linux can do all this and save time on downloading patches and updates for multiple computers, and you can even get a faster surfing experience on frequently visited sites. You can run it on old or new hardware, and in fact a nettop with a USB adapter might be the ideal way of doing this from a power perspective. And all of this is free, assuming you have the necessary hardware. Pointing out flaws we already list in the article (i.e. the power concern) is a waste of time. I put it as the last sentence figuring that if nothing else, people would read the conclusion and see our discussion of power concerns.
  • michal1980 - Tuesday, May 11, 2010 - link

    I get your point Anandtech guru's. And the article is fine. But it seems like you guys are deaf right now.

    For most users, even power users, the question remains, why? What REALLY benefits will I see for all this new up keep.

    IMHO, for a home user this proxy is equal to the killer nic. Might work, but the money at the end of the day is better spent elsewhere.
  • dezza - Wednesday, May 12, 2010 - link

    With an Atom PC or any small form factor PC that has at least 1GHz or whatever depending on the services you will be running - You will be better left off with combining DHCP/Proxy so you have one connection open always to gateway/proxy .. And instead of auto-detect explicitly define it ..

    http://www.broadband-help.com/articles/networking/...

    I found this ..

    Brings a few points into the light once again .. Static content is the only thing that is affected, which is of course a big part, but since many big sites uses systems like imageGet()'ers etc. in PHP/ASP and thumbnail() functions - Your proxy can't touch this (MC Hammer) ..

    Again .. Chris, I respect your article and I agree that ArchLinux is a great distribution (In my case for bleeding-edge workstation) - I love reading anandtech's hardware articles as well and this is the main reason for having it in my feeder, but I will patiently wait while more of these articles get to the surface so we give feedback and maybe even come with suggestions or help you in forging them .. Would be lovely to extend this site with some killer articles on software/programming etc. I never doubt your quality of hardware articles and I think indeed you wrote a decent article. This is no bashing.
  • dezza - Wednesday, May 12, 2010 - link

    http://tools.ietf.org/html/rfc3143

    another official rfc documenting problems with the proxy ..

    Not even on my work where we have 8000 clients connected to the internet and using BitTorrent heavily (We have BitTorrent shaping/filtering with encryption support) we would benefit anything from using a proxy.

    Also with a proxy you will have to scale your proxy tremendously with another 1000 users I/O performance of the proxy server drops incredibly ..
  • jamyryals - Tuesday, May 11, 2010 - link

    They are linux experts. This means they know too much to actually read the article.

    Jarred, you are on point with the distro v distro comment.
  • eleon - Wednesday, May 12, 2010 - link

    "Do you have a growing family at home slowly eating away at your bandwidth? Maybe you're a web surfing fanatic looking for a little more speed? If you answered yes to either, a caching proxy is for you."

    That the first paragraph of this article and that's the first thing readers will see. and I really doubt that a caching proxy is the right solution. A caching proxy won't help if one client use the whole bandwidth with bittorent. It will only have a benefit if you have multiple downloads of the same static (http or ftp) content, and that's not the scenario families are dealing with. And if you really have some big updates or Servicepacks, so why not only downloading them one and share between the client. So this maybe a solution for you special needs, but obviously not for a "normal" family. So it's right that you didn't mentioned QOS, but if someone is eating away your bandwidth you need QOS!

    and replied to my comment:
    "I'm not worried about "saving bandwidth" in the way you're discussing, though if you had a plan that charged you for downloading over a certain amount it might be useful. I'm interested in speeding up patching and such."
    PLEASE differentiate between"bandwidth" and "transfer-volume". I didn't talk about saving bandwidth, I said that proxyserver can be a solution for reducing the transfer-volume, that is something completly different. As long as you don't distinguish between this two things you will never understand what you can do with QOS, and what you can do with a caching proxy.

    and I didn't start a discussion about distribution X is better than distribution Y.
    I really love Archlinux.
    But said Archlinux is a good choice for this proxy, because it has a rolling release cycle.
    My comment about archlinux only relies to this, because it shows that you have no idea about the advantages of distributions with stable releases (+security updates), and releases with rolling release cycle. And in my opinion it is really irresponsible to recommend a rolling release distro for a router/firewall/proxy. (the reasons for that are in my first post).

    So please don't get me wrong, if this is satifying your needs, it's perfect and I'm happy for you.
    but if someone can answer your questions "Do you have a growing family at home slowly eating away at your bandwidth? Maybe you're a web surfing fanatic looking for a little more speed?" with Yes, he or she wouldn't be happy with a caching proxy. It isn't a direct solution for this, it will maybe help in a indirect way in some special situation if you download the same big files by http (ftp) multiple times.
    So if you answer this questions with "Yes" you should consider QOS.

    My main concern is, that your solution is not effective! You can improve the efficiency by low-power or even embedded hardware, a special router-distribution which will minimize the setup time, so it will be efficient on multiple levels, but it won't change the fact, that it is really ineffective in solving the problems of "eaten bandwith", and slow websurfing experience.

    Running QOS an embedded hardware would be effective and efficient. (and many SOHO- and even consumer-routers support it out of the box, and if not, many are supported by alternative firmware-distributions like openwrt, dd-wrt,... ) so if you have these problems/needs, this probably would be the way to go.
  • JarredWalton - Wednesday, May 12, 2010 - link

    But QoS won't give you more speed, it will just prioritize bandwidth. A caching proxy, on the other hand, can actually boost page load speeds a lot (though not always). It's not for everyone, and I suppose part of the problem is I view things from my world while Chris has his own idea on things. Anyway, you're still getting caught up on what is essentially a hook to the article. Read that paragraph this way, and it's just a less dramatic restatement of Chris' paragraph:

    "Do you find your web surfing experience to be slower than you'd like? Do you have lots of PCs and do you frequently download the same file on multiple computers? If so, you might want to consider reading this article about proxy servers and what they can do, because it might be something that will help alleviate some of your bandwidth congestion."

    Call it over-exuberance on the part of the author or whatever. Just because someone likes the idea of proxies and writes an article -- OMG it's on AnandTech so it must be true! -- doesn't mean it's the right solution for every single situation. Given this is a Linux article, I personally thought it was more of an interesting idea that may be useful to some of our readership. I know the caching of Windows Updates is definitely useful for me, even though I have a relatively fast 16Mbit download speed.
  • epi 1:10,000 - Tuesday, May 11, 2010 - link

    It would be nice if someone could review a realtime av scanning proxy w/ caching. Has anyone tried SafeSquid, or dansguard squid w/ clamav?
  • SquattingDog - Tuesday, May 11, 2010 - link

    This is a great article, and comes in a very timely fashion, as I am looking to set something like this up in our flat. We have a 20GB monthly cap, and need to distribute the per-GB costs out to each person based on their usage and possibly limit their usage if they exceed 5GB for example. Is this possible with a Linux Proxy or QoS tool? If so, what should I be looking at to do this - and are there any which are quick and easy (in relative terms) to set up? (I am a Linux noob atm)

    Second question has to do with latency for games. One of the people in our flat plays games like Bad Company 2 online a lot of the time. That, basic browing and MSN are the only things he uses the internet for. What is the added delay with a transparent proxy in place for gaming? I know you mentioned Steam updates not working with proxy caching during the article, Jarred, but what about the gaming itself? Is there a measurable/noticeable latency increase? An increase in the order of 2 - 5ms is acceptable, and we can always get interleaving turned off on our line to mitigate this.
  • ChrisRice - Tuesday, May 11, 2010 - link

    With the setup mentioned in the article you will have no adverse effects to your gaming. I will look up your proxy quota question, I believe there are a bunch of solutions available.
  • SquattingDog - Tuesday, May 11, 2010 - link

    Thanks Chris, that would be outstanding!
  • JarredWalton - Wednesday, May 12, 2010 - link

    Yeah, I tested gaming and didn't notice any problems with the proxy. Steam works fine BTW, but it doesn't go out through the proxy so the updates aren't cached. (I tried sending the Steam update ports through the proxy but then Steam wouldn't connect... looking around online, numerous folks are saying Valve doesn't allow use of Steam through a proxy.) Bad Company 2 also works fine, as do quite a few other titles I've played.

    Squid can do a lot of things not discussed in this article, but how well it does them and how easy they are to configure is probably something for a follow-up. As something of a Linux router newbie myself, I'm not quite sure how you go about restricting access and putting download caps on the various clients, but the squid.conf file suggest all of that is possible.

    I'll leave the rest to Chris. :-)
  • SquattingDog - Wednesday, May 12, 2010 - link

    Thanks for coming back to me on this Jarred, great news for myself and my other flatmates then - I wouldn't be popular if suddenly everyone's ping went up 30 - 50ms ;)
  • mariush - Wednesday, May 12, 2010 - link

    Get a managed switch and use MRTG (http://en.wikipedia.org/wiki/Multi_Router_Traffic_... or Cacti or other solutions to log how much traffic each port does.

    With a proxy, you'd have to create a username and password for each member in your house or log traffic on the server based on MAC address or IP which is a bit more complicated than simply polling the switch with such software and logging the bytes transferred.

    See here some managed switches http://www.newegg.com/Product/ProductList.aspx?Sub...

    Though there may be cheaper unmanaged switches which have SNMP feature, the thing you need for logging traffic.
  • SquattingDog - Wednesday, May 12, 2010 - link

    That is a good solution too, however I have existing hardware lying around that I could put to use for the Linux box, and that would require me shelling out for both a new switch and a new wireless router (everyone but me connects via wireless, and it's an all-in-on Netgear DG834G ADSL Modem/Router) - and our pricing here is not as good as yours over there, unfortunately :(
  • SquattingDog - Wednesday, May 12, 2010 - link

    This DG834G is v5, so I'm SOL for enabling built-in SNMP, as it doesn't support that. The Proxy server set up would work well for us also, as I frequently have to download Windows updates on various machines which come and go, and having them locally cached will reduce internet usage substantially. :)
  • mindless1 - Tuesday, May 11, 2010 - link

    Some of you talked about cost or especially power savings. Has it occurred to you that you can run a proxy on the windows box you probably already leave running most if not all the time and expect a trivial increase in power consumption from doing so?

    Sure, you'll need have the amount of memory you want to devote added over the amount your system would otherwise need, but in this day and age of multi-gigabyte endowed systems it isn't much to devote 1/4th your memory to the job... if you really need that much which many people won't.
  • JarredWalton - Wednesday, May 12, 2010 - link

    I looked around at various options, but for the free stuff it appears that you'd need to manually configure each browser to go through the Windows proxy (i.e. instead of having a transparent proxy). Anyway, my Windows machines are all even more power hungry than my test proxy, so I don't leave them running at night. But I believe squid is even available for Windows platforms:
    http://wiki.squid-cache.org/SquidFaq/BinaryPackage...
  • Jeff7181 - Tuesday, May 11, 2010 - link

    A Pentium 4 3.06 GHz chip with HT? Are you insane? You're going to use one of the most power hungry consumer grade CPU's of all time to run a firewall & proxy? I'm all for dinking around with old hardware and turning them into a Linux box... file server... router... whatever. But a 3.06 GHz P4???

    Regardless... I like this... in fact, I'd like you see you add DNS and Samba to the build. (hey... if you can afford the power draw of a P4, what's another 20 watts for a couple high capacity hard drives?)
  • mariush - Tuesday, May 11, 2010 - link

    It would have been much easier to install FreeBSD and squid... it's just a question of adding pre-made packages that configure and install by themselves.
  • GullLars - Wednesday, May 12, 2010 - link

    Thanks a lot for this guide. It will get a bookmark for later use.
  • Brian B - Wednesday, May 12, 2010 - link

    I have a VM host system in the basement. It would be very easy to bring up a tiny VM with two virtual NICs and install LINUX. But does anyone know if this setup should function correctly? Since the VMs are in most every way just PC's on the network, I cant think of a good reason it *shouldnt* work but maybe I'm missing something...
  • ChrisRice - Thursday, May 13, 2010 - link

    Yes it will work.
  • CZroe - Thursday, May 13, 2010 - link

    I want to set up OpenVPN to allow me to route my Internet traffic through a certain PC from another Internet connection. I can think of many uses. For example, if I want to hide tethering traffic from my cellphone provider, I can open a an encrypted VPN tunnel using a VPN client on the tethered PC to connect to my home network's VPN server but, instead of just using it to access remote files and LAN services, I want to use it to route Internet traffic through it (through VPN connection over Internet to LAN and back out onto the 'net). I know this is possiblem I just don't know how.
  • thebeastie - Thursday, May 13, 2010 - link

    I have always liked the idea of proxy servers, but the problem I have always seen with them is that people who set them up only set them up to proxy small content like web images etc and ignore some one watching a web cast of CNN or something.

    maximum_object_size 2048 MB
    ? Why bother.

    Web proxies were huge in the 90s but died away as general internet got fast or people just unfairly blamed or assumed the proxy is ruining their internet under any circumstance where the internet wasn't working.

    I think there is a bigger future in 'large content only' proxy servers where if something is more then 8megs, then cache it.

    I would say proxies were started when web images were the biggest thing on web pages in the 90s but that has turned full circle and is now the smallest thing on the internet.

    The other crazy part of it all is that a lot of web sites that have only large content like news videos go out of their way to make sure it cant be cached, why bother.
  • Kenazo - Thursday, May 13, 2010 - link

    Not sure if it's been mentioned in any of the comments yet, but I've used Smoothwall Express with good results for home and small business use. Linux based, easy to install and can run on any garbage Pentium III you have kicking around.
  • pkoi - Sunday, May 16, 2010 - link

    +1, "easy to install and can run on any garbage Pentium III you have kicking around. "
  • gwolfman - Thursday, May 13, 2010 - link

    I'd go the transparent method, except for the fact that the Netflix plugin/addon for WMC (Windows Media Center) gives me a bogus error. Without the proxy, all works as is should. Anyone else run into this?
  • Lio - Saturday, May 15, 2010 - link

    Would be interesting to have some focus on cost of running this setup. Linux may be free but running a machine 24 hours a day to provide a proxy service is not. Older hardware such as that based on Intel's Pentium 4 chips are famous for high power consumption, I wonder what the additional charge on the power bill would be after one year. Is there anything that can be done to reduce the operating cost of such a set up via software which supports some kind of sleep mode?
  • pkoi - Sunday, May 16, 2010 - link

    One way to curb power consumption is to do meaningful computation on idle time 24/7.
    BOINC ect. And run linux in a VM.
  • RoboJ1M - Sunday, May 16, 2010 - link

    Hi,

    A bit late in the day but I'd like to point everybody's attention to "eBox", a web based configurator for linux based server.

    It combines the ease of use of the "home router" with the tremendous power of the linux server and I absolutely love it.

    We use two of them at work for proxy, firewall and vpn duties and I use one at home for all of the above and domain, print and file services.

    For reference it's based on ubuntu server (8.04 LTS at the mo but 10.04 coming soon)

    Anyway, I absolutely love it. :)

    J.
  • zabby113 - Monday, May 17, 2010 - link

    Does the proxy take over as your router? Since it leases your IP from the Modem, is this not where port forwarded and such should take place? If so, how does one perform such tasks? If not, can anyone explain to me what is happening?
  • Fineghal - Monday, May 17, 2010 - link

    This seems like the perfect task for a "plug" computer ala GuruPlug. 2Gb Ethernet, 1 eSata, 2 usb and b/g wireless. 130 USD or so - if you want to do this, the power savings of 5 watts vs 100 watts comes close to paying for itself in a year. Obligatory link: http://www.globalscaletechnologies.com/p-32-gurupl...
  • jtleon - Tuesday, July 20, 2010 - link

    I realize Anandtech probably will not touch this subject - but lets face it - Ads eat up bandwidth. More than we may EVER want to admit.

    Using a proxy that has filtering built-in, like CCProxy, can GREATLY eliminate all those hungry Ads, and save that precious bandwidth for what you REALLY need it for - Useful data, of course!

    We use CCProxy both at work, and at home, and build our webfilter using the updated host file from:

    http://www.mvps.org/winhelp2002/hosts.htm

    This way not only do we eliminate redundant downloads, but we stop the Ads - before they hit the router! Talk about truly efficient bandwidth!!!

    jtleon

Log in

Don't have an account? Sign up now