Proxies: About, Setting up with Mac OS X

[Ledgem]

Greetings all!

It seems I'll be working my old job for a few months more, meaning that I'm going ahead with a network overhaul for our task. Basically, all systems are linked via a gigabit switch, which is really incredible - systems that had only FireWire 400 ports are now able to access our FireWire 800 drives at full speed when shared from a computer that has FireWire 800 ports (all (Mac) systems, except for our Dell XPS, have gigabit ethernet). One of our systems has two NICs as well - I'd originally intended for this system to act as a gateway-type of system for the network so that all systems could still retain net access.

There is a slight problem that occurred, however. University policy forbids us from using routers, or attaching more than one machine to a port (had this policy not existed I would have requested a router instead of a switch). I figured that this wouldn't be a big issue and planned to use Mac OS X's "internet sharing" to get around this - the university was never able to detect my dorm router, after all, and from what I knew internet sharing should have simply forwarded all communications and made it look like they were coming from the one sharing machine.

While the internet sharing worked, for some reason the sneak aspect didn't work, and we were caught. I'm now wondering if I shouldn't try to configure the systems to use the gateway as a proxy, and if that might make a difference. First, does anyone have any suggestions about the feasability of that idea, or any other ideas? If it's good, then how should I go about setting it up (external software probably required)?

Note that this isn't urgent. After a Quicktime update screwed over some software we use I put a freeze on our updates. The internet is largely for me to diagnose problems, research efficiency and new methods, and of course burn some time with. I've set up VNC for relaying control across the systems (what I'm doing now to type this out) so I'm not out of luck, either. I'd just rather access directly and not worry about this somewhat sluggish (even over gigabit) VNC.

Thanks in advance

[Epyon9283]

You'd have to proxy everything to avoid getting detected. squid is a good http proxy. You may be able to use a socks proxy but honestly I don't know what the traffic coming out of a socks proxy looks like and whether or not it would be easy to detect.

[SeijiSensei]

Tell them that you're working on a legitimate project and that either you need an IP subnet so that each of your machines can have an address on their network, or that you're going to use a NAT device. Really how can they ban routers from the network? Maybe it makes some sense in residential areas, but not in a legitimate activity like yours.

If you were using some ordinary router like a Linksys or Netgear, they might have discovered it by looking for "signatures." The nmap program (http://www.insecure.org/) can do this for many devices. If the university won't play ball, I'd stick a Linux box there configured as a NAT router. From their perspective, all they'll see is a Linux box; its purpose shouldn't be that obvious.

The OS X box should have looked equivalent to this solution. Perhaps it wasn't fully firewalled? I'd only open inbound ssh and block all other inbound traffic on the external interface. You're mostly interested in outbound connections, I would think. A fully-firewalled NAT router should be pretty difficult to discover even with powerful tools like nmap.

Are these the kind of policies university network admins apply these days in an wrong-headed attempt to stop "piracy?" Or are they worried that if you connect a router, you'll mess with their IP address allocations? A NAT router with a private network behind it will have zilch effect in the latter case. Perhaps they don't want people spreading their bandwidth around wirelessly. If so, what's wrong with a normal wired Ethernet router?

These days even a plain-vanilla Windows XP box can act as a router (using "Internet connection sharing").

[Epyon9283]

Originally Posted by SeijiSensei

If you were using some ordinary router like a Linksys or Netgear, they might have discovered it by looking for "signatures." The nmap program (http://www.insecure.org/) can do this for many devices. If the university won't play ball, I'd stick a Linux box there configured as a NAT router. From their perspective, all they'll see is a Linux box; its purpose shouldn't be that obvious.

The OS X box should have looked equivalent to this solution. Perhaps it wasn't fully firewalled? I'd only open inbound ssh and block all other inbound traffic on the external interface. You're mostly interested in outbound connections, I would think. A fully-firewalled NAT router should be pretty difficult to discover even with powerful tools like nmap.

All they have to do is sniff the traffic coming out of that gateway machine to ascertain whether or not its acting as a NAT device. A really easy way to tell is by looking at the TTL of the packets coming out of the NAT box. They'll be decreased by one when they shouldn't be. OS X's default TTL is 64. After a packet traverses the gateway it'll be 63.

[SeijiSensei]

Originally Posted by Epyon9283

A really easy way to tell is by looking at the TTL of the packets coming out of the NAT box.

Hmm, that sounds like something an open-source firewall could be patched to fix pretty easily. I suppose automated scanners make this kind of snooping possible, but sniffing all the traffic from thousands of machines looking for this pattern seems like a lot of effort for little reward.

That makes an application-level proxy like squid more attractive since it doesn't muck with the packets. Squid would be fine if you're only interested in using HTTP.

[Ledgem]

Originally Posted by Epyon9283

You'd have to proxy everything to avoid getting detected. squid is a good http proxy. You may be able to use a socks proxy but honestly I don't know what the traffic coming out of a socks proxy looks like and whether or not it would be easy to detect.

Thanks for the recommendation, I'll look into Squid.

Originally Posted by SeijiSensei

Are these the kind of policies university network admins apply these days in an wrong-headed attempt to stop "piracy?" Or are they worried that if you connect a router, you'll mess with their IP address allocations? A NAT router with a private network behind it will have zilch effect in the latter case. Perhaps they don't want people spreading their bandwidth around wirelessly. If so, what's wrong with a normal wired Ethernet router?

I don't believe it's about piracy so much as about network security. I was actually surprised to find that the university doesn't engage in filtering websites or programs - they didn't when I was a freshman, and they still don't. If the network detects that your system is scanning other systems, it'll shut off the port you're connected to and report your system as infected. I believe your MAC address is also blacklisted. You'd need to go to IT and tell them that your system is clean to get the block removed. (Hooray for MAC address changing to get around that.) If it happens three times, IT comes to you and formats your computer if you want network access.

It's not a terrible policy in that regard. One of my roommates was once infected with something (business majors...) and I woke up to find my connection shut down - his computer had sent out 10,000 spam emails through my wireless connection. The network caught it and shut it off. I wasn't pleased that I was dragged into it, as they saw it from a MAC registered under me, but I was pleased that they'd cut off some spam.

Anyway, back on topic. The connections we use at work are a bit different than the regular university network. Regular connections are 10 mbps links and are subject to bandwidth regulations (limits are ~3 GB (up+down) in two hours, or 10 GB per 24 hours - quite generous overall). We're running on 100 mbps without restrictions, I think. Also, all of our systems have designated IPs, whereas the regular university lines are all DHCP. If I may rant against the IT services here... our systems are routed through a Blade server, but it's not exclusive to us and I believe it's overloaded. We used to get speeds lower than DSL until they finally listened and took some of the load off of the server. But we still don't get anywhere near 100 mbps speeds (it gets better at night, of course), which is why I pushed for us to link all of our systems directly.

I did ask IT if we could put a router on or use some other solution, but they're pretty adamant about not having more than one device on a jack. It's true that it makes it harder to determine what device was breaking regulations or has become compromised, and I'd imagine that they deal with a ton of clueless users. Such a regulation probably makes it a lot easier on them.

Either way, proxying won't allow for detection by the TTL count, right? I'll need to be sure that I set it up properly. I think it may be easier to do it through OSX than through the WindowsXP system - I remember trying to proxy through Tor under XP, and every single program had to be configured individually to go through the proxy. I'd guess some of the OSX programs will have to be configured for that too, but all I really need is a web browser.

[SeijiSensei]

If all you want is web services, squid's probably the best solution. If you Google around for "transparent proxy" you can learn how to configure a Linux box with iptables to intercept outbound HTTP requests and push them through squid. If you're only going to proxy a couple of machines, it's easier to configure the client browsers to use the proxy than set up transparent proxying. As the number of clients grows larger, transparent proxying becomes more appealing since you don't need to configure each individual browser.

If you're concerned about your privacy, though, you'll want to delete the squid logs regularly since they'll contain a history of every single object anyone using the proxy has requested.

Squid also has an elaborate system of access controls. I use these in Windows shops to block downloads of executable files by ordinary users. I also have rules to intercept requests for objects on common advertising and malware sites. Again something like the AdBlock Plus plugin for Firefox can block ads on the client, but you're again faced with installing the plugin on the individual machines or rolling out a custom Firefox build to the workstations.

[Ledgem]

Originally Posted by SeijiSensei

If all you want is web services, squid's probably the best solution. If you Google around for "transparent proxy" you can learn how to configure a Linux box with iptables to intercept outbound HTTP requests and push them through squid. If you're only going to proxy a couple of machines, it's easier to configure the client browsers to use the proxy than set up transparent proxying. As the number of clients grows larger, transparent proxying becomes more appealing since you don't need to configure each individual browser.

If you're concerned about your privacy, though, you'll want to delete the squid logs regularly since they'll contain a history of every single object anyone using the proxy has requested.

Transparent proxying sounds very neat, but for six systems (not including the proxy) it's probably easier to just configure each with proxy settings. I'd really just like to be able to browse the web and potentially patch the systems. I haven't looked into Squid yet, but is there a way to disable logging? Worst case scenario I can create an automated routine that wipes the log every 1 hour I suppose...

[SeijiSensei]

Sure, point all the logging entries in squid.conf to /dev/null. You might be able to turn logging off entirely from the configuration file as well. Squid has dozens of settings; I only pay attention to a few.

[Ledgem]

I'm currently building Squid and a few other tools to make configuring it easier (I'm not a CLI guy), and I was just wondering - are there any security precautions that I should take with Squid? Reconfigure the default port, or anything of that sort? Since it's just a proxy service I wouldn't imagine that it should represent a security issue, but I'd rather be safe than sorry (especially since there is one $5000-configured Windows box on the network).

[Epyon9283]

Assuming that squid will be running on the gateway, ensure that it is listening only on the interface sitting on the inside. If you really want to, force authentication to use the proxy. Squid supports basic and ntlm auth IIRC (there may be other auth methods).

There are a lot of options for ACLs in squid. They allow you to restrict access to content if you choose to do so. By default they only allow http traffic on certain ports. I had squid set up to send traffic through clamav to protect the clients from viruses. It was a pretty cool setup.

[Ledgem]

Epyon9283 have you done the setup of Squid on a Mac OSX box? I've run into some odd problems, and I'm still at the stages of installing some back-end stuff.

I'm following the instructions from Macports. I already had Xcode Tools installed, and then installed Macports. Preferring GUI to CLI, I installed Porticus (mentioned on Macports) and began installing what they list as necessary for Webmin (for easier configuration of Squid). Porticus successfully installed Perl, but gets stuck at "Configuring" p5-SSLeay-something. I've twice tried to install it, and it always gets stuck, chewing up a lot of CPU while it's at it. I figured I'd try to install via commandline, but I just get the error "port: command not found."

Any ideas? Switch over to Darwinports, perhaps?

[SeijiSensei]

As yucky as it sounds, I'd vote for editing /etc/squid/squid.conf (or wherever it lives on an OS X system) manually and forgetting about all that GUI stuff. There really aren't many things to set, mostly just the appropriate "acl" lines to enable access from the internal network.

If you have a working binary, and can edit squid.conf, I'd go that route. Once you've configured it, you'll probably never need to futz with it again, especially in the context you're using it for.

[Epyon9283]

Darwinports is just what Macports used to be named. I would avoid webmin at all costs. I have had it screw up my config files a few times in the past.

Squid only has a couple dependencies:

Mac-Pro:~ tom$ sudo port deps squid
squid has library dependencies on:
	openssl
	zlib

I already had both installed so squid took less than 3 minutes to download and compile on my Mac.

The squid.conf file (actually located in /opt/local/etc/squid/) is actually quite well documented by the comments in the file. Its not terribly difficult to figure stuff out if you read the documentation. For instance, searching for listen in the config file brings you to about of page of documentation about the http_port option. This is where you would specify the address and port you want to listen on.

[Ledgem]

I suppose I'll give the CLI a try. But I'm still getting the "port: command not found" error, so I can't even install Squid through there. I've tried reinstalling Macports but it didn't fix it. I've enabled the Root user and all, too. Not sure how to get around that one...

[SeijiSensei]

Maybe the port command isn't in your path? Try using its full path from the prompt, something like /usr/sbin/port perhaps?

In Linux, if you log in as an ordinary user, then su to become root, you inherit the user's environment. Most users don't have the sbin directories in their path because they're not supposed to be running programs in those "superuser" directories. Usually I just add /sbin:/usr/sbin:/usr/local/sbin to the path in my user profile so they're available if I become root. For the bash shell, PATH is usually set in .bash_profile or, sometimes, .bashrc. Other shells have different files. csh uses .cshrc, etc.. Try a 'grep PATH .*' command your home directory to search for the string PATH in all files prefixed with a period. That should show you where it's set.

[Epyon9283]

Your ~/.profile should contain the first line here if macports properly installed:

export PATH=/opt/local/bin:/opt/local/sbin:$PATH
export MANPATH=$MANPATH:/opt/local/share/man

If it doesn't, add those lines. It makes it a lot easier to use anything you install via macports. The MANPATH line gives you access to the man pages of the stuff you install via macports.

[Ledgem]

Originally Posted by SeijiSensei

Maybe the port command isn't in your path? Try using its full path from the prompt, something like /usr/sbin/port perhaps?

I tried - it wasn't there, either. I tried the GREP command but it didn't do anything - I must be using it incorrectly.

Originally Posted by Epyon9283

Your ~/.profile should contain the first line here if macports properly installed:

I guess I have to admit that I'm still relatively new to the Unix system structure and operations. And I see that DOS commands don't really work here, either... I had a "dictionary" of Linux commands that I'll have to look up again, but how do I access the .profile file?

Sorry for the delays on my part, the department just bumped me up to Final Cut Studio 2 (woohoo!) so now I'm focusing most of my energy on trying to set up distributed encoding through Qmaster and re-work the workflow to go through Compressor. I really would like to get this proxy issue figured out, though. Thanks for the help so far, and apologies for my overall helplessness in CLI land!

Edit: I did some more work with it and found that this system - the only system down here that I haven't had to do a full system reinstall on - didn't have X11 installed. So I installed X11 and then reinstalled MacPorts (well, the installer said it was an Upgrade), but I'm still not able to use the 'port' command from the terminal. I did quit and reopen the Terminal after installing X11, too. I've updated X11 and it didn't make a difference.

[Epyon9283]

If the output of "echo $PATH" doesn't look like the following then you need to change your ~/.profile

Mac-Pro:~ tom$ echo $PATH
/opt/local/bin:/opt/local/sbin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/X11/bin

To edit the file in the terminal, type the following:

nano ~/.profile

You should see something like this:


Enter in the PATH and MANPATH lines I posted earlier. To save and exit, hit ctrl+x and then y to save.

Then after doing that, either close the terminal and go back in or type "source ~/.profile" to update your paths in your existing session.

Once you do that you should have access to the port command. You'll need root access to use the command though so run it with sudo. For example "sudo port install squid."

[Ledgem]

Thanks Epyon, that seems to have done it. I didn't even have a ~/.profile, apparently, but I put it in and now the PORT command works. I guess I was lacking the OpenSSL dependency, so I attempted to get it and hit an error; the program requested that I run 'port clean openssl' so I did, and now I'm reinstalling it. It's currently building. Once that's done I'll go for Squid. Hopefully configuration options won't be too difficult...