Question:
I am looking into finding a way to track public bandwidth usage on a per-instance or per elastic IP basis. Amazon does not seem to offer these metrics. You can get total in/out bandwidth through their reporting mechanisms, but this includes private network bandwidth, and is account wide. You can use cloudwatch to gather more in depth metrics, but they also lump public and private bandwidth together. We are looking into rolling our own, but your servers are built with one interface, and any elastic IPs are NAT’d to that interface. Since everything goes through one interface, it is all lumped together.
Does anyone have any suggestions? Have you ever encountered a similar issue? That is a linux server environment with one interface from which you had to determine public bandwidth usage.
Answer:
Answering an old question for the benefit of Googlers.
We encountered a similar problem, and “solved” it using iptables counters, making us of the fact that all outgoing traffic that is private will be on a 10.0.0.0/8 IP address, with the remainder being public traffic. You can also track input for other purposes; only outgoing public traffic is charged, of course.
So, create some counters:
|
1 2 3 4 5 |
iptables -A INPUT -s 0.0.0.0/0 --> Total incoming traffic iptables -A INPUT -s 10.0.0.0/8 --> private incoming traffic iptables -A OUTPUT -d 0.0.0.0/0 --> Total outgoing traffic iptables -A OUTPUT -d 10.0.0.0/8 --> private outgoing traffic |
Check counters:
|
1 2 3 |
iptables -nv -L INPUT --> counters about incoming traffic iptables -nv -L OUTPUT --> counters about outgoing traffic |
NOTE: When you use the values, you get private and TOTAL: so to get public, subtract private from Total before using it for anything.
You can also zero out the counters if you don’t want to report cumulative bandwidth:
|
1 2 3 |
iptables --zero INPUT --> clear counter iptables --zero OUTPUT --> clear counter |
The following is an (ugly) bash script that will push this information out to Ganglia, assuming you created the counters already:
|
1 2 3 4 5 6 7 8 9 10 |
#!/bin/bash OUTPUT_PUBLIC=`sudo iptables -nvx -L OUTPUT | head -3 | tail -1 | tr -s [:blank:] |cut -d' ' -f3` OUTPUT_PRIVATE=`sudo iptables -nvx -L OUTPUT | tail -1 | tr -s [:blank:] |cut -d' ' -f3` let OUTPUT_PUBLIC=$OUTPUT_PUBLIC-$OUTPUT_PRIVATE sudo iptables --zero INPUT sudo iptables --zero OUTPUT gmetric -n "public_outbound_traffic" -v $OUTPUT_PUBLIC -t uint32 -u "bytes" gmetric -n "private_outbound_traffic" -v $OUTPUT_PRIVATE -t uint32 -u "bytes" |
Run this in a cronjob, just make sure that the cronjob frequency matches up with your ganglia reporting frequency (or otherwise handle possible mismatches).
Hope this helps someone.