SIP Testing Basics

This is a walk through on what it took to get a single SIP load test generated, using SIPP.  Getting the first one was hard, so I wanted to document these steps in case it helps anyone else, and for future reference.

Tools

There’s not a lot of SIP load tools out there.  For SIP load, there’s really one standard that keeps coming up in discussions: SIPP.  SIPP, which is at: http://sipp.sourceforge.net/ is a command line tool that drives various types of SIP traffic to a proxy. There’s a lot of potential with this tool, it actually is highly configurable and has quite a few amazing capabilities, as seen in their documentation (http://sipp.sourceforge.net/)

SIPP is also one of the most difficult things to learn. If you are new to Session Initiation Protocol (SIP) and just starting to test it, learning about SIP and SIPP is pretty daunting.  However, I have one advice, stick with SIPP.  I’ve tried every open source tool out there, and for SIP load testing, SIPP is by far the best.

After I got SIPP scripts up and running, I then started integrating them into Jenkins, and letting Jenkins maintain the scripts and email me the results of each run (more on that in a separate blog post: http://www.continuous-qa.com/2013/02/sipp-and-jenkins.html

SIP Basics

SIP stands for Session Initiation Protocol.  I won’t pretend to be an expert on it.  I was thrown into the world of SIP as a need arose for SIP Load testing where I work.

Basically, SIP is a protocol that handles the handshaking and transfer of data for VOIP services… Voice over the Internet.  But this isn’t limited to just handling tools like Skype, Teamspeak, etc.  In fact most phone calls, at some point, become SIP calls.  You may pick up a hard line, dial a number, it could route to your PBX and at some or many points in the transaction go to a SIP PBX.

SIP is a standard of packet protocols. It requires certain things to occur in specific sequences.  There is room for some variance, but in general everything works within a standard format.

Simple Call

In a simple call where one phone dials another, and SIP is being utilized, we’d see something like this occur:
12136667777@someplace.com INVITE —–> 1000@somepbx.com
100 Trying <-------- Phone 2
200 OK <----------- Phone 2
ACK —————->

In the above example there’s some similarities with the Web world… first off the Sip addresses look like emails: 12136667777@someplace.com and 1000@somepbx.com

That’s how SIP accounts are registered. The first is an example of a potential phone number… listed on a proxy or pbx.  the other is an example of extension 1000 on a pbx.

The responses also look like HTTP status codes… that’s because they are related.  A list of SIP status codes can be found here:
http://www.websitepulse.com/kb/sip_status_codes.html

You’ll notice a lot of similarities with HTTP codes… 400 range are errors as are 500, 200’s are usually “OK’s” and so forth.

Details of a Simple Call

Using Wireshark or other packet sniffers you can grab the packets and see what they look like. Below are some examples of the packets from my local network:

Invite

———————————————– 2013-02-05 16:52:14:961.441
UDP message sent (505 bytes):

INVITE sip:1000@10.98.6.26:5060 SIP/2.0
Via: SIP/2.0/UDP 10.98.7.36:5061;branch=z9hG4bK-1769-1-0
From: sipp ;tag=1769SIPpTag001
To: 1000
Call-ID: 1-1769@10.98.7.36
CSeq: 1 INVITE
Contact: sip:sipp@10.98.7.36:5061
Max-Forwards: 70
Subject: Performance Test
Content-Type: application/sdp
Content-Length:   131

v=0
o=user1 53655765 2353687637 IN IP4 10.98.7.36
s=-
c=IN IP4 10.98.7.36
t=0 0
m=audio 6000 RTP/AVP 0
a=rtpmap:0 PCMU/8000

100 Trying

———————————————– 2013-02-05 16:52:14:961.782
UDP message received [332] bytes :

SIP/2.0 100 Trying
Via: SIP/2.0/UDP 10.98.7.36:5061;branch=z9hG4bK-1769-1-0;received=10.98.6.26
From: sipp ;tag=1769SIPpTag001
To: 1000
Call-ID: 1-1769@10.98.7.36
CSeq: 1 INVITE
User-Agent: FreeSWITCH-mod_sofia/1.3.13b+git~20130204T173656Z~f3206e5ff1
Content-Length: 0

200 OK

———————————————– 2013-02-05 16:52:15:2.033
UDP message received [1066] bytes :

SIP/2.0 200 OK
Via: SIP/2.0/UDP 10.98.7.36:5061;branch=z9hG4bK-1769-1-0;received=10.98.6.26
From: sipp ;tag=1769SIPpTag001
To: 1000 ;tag=cUcc2K9Z5Ut1F
Call-ID: 1-1769@10.98.7.36
CSeq: 1 INVITE
Contact:
User-Agent: FreeSWITCH-mod_sofia/1.3.13b+git~20130204T173656Z~f3206e5ff1
Accept: application/sdp
Allow: INVITE, ACK, BYE, CANCEL, OPTIONS, MESSAGE, INFO, UPDATE, REGISTER, REFER, NOTIFY, PUBLISH, SUBSCRIBE
Supported: timer, precondition, path, replaces
Allow-Events: talk, hold, conference, presence, dialog, line-seize, call-info, sla, include-session-description, presence.winfo, message-summary, refer
Content-Type: application/sdp
Content-Disposition: session
Content-Length: 187
Remote-Party-ID: “1000” ;party=calling;privacy=off;screen=no

v=0
o=FreeSWITCH 1360084930 1360084931 IN IP4 10.98.6.26
s=FreeSWITCH
c=IN IP4 10.98.6.26
t=0 0
m=audio 27004 RTP/AVP 0
a=rtpmap:0 PCMU/8000
a=silenceSupp:off – – – –
a=ptime:20

ACK

———————————————– 2013-02-05 16:52:15:2.159
UDP message sent (351 bytes):

ACK sip:1000@10.98.6.26:5060 SIP/2.0
Via: SIP/2.0/UDP 10.98.7.36:5061;branch=z9hG4bK-1769-1-5
From: sipp ;tag=1769SIPpTag001
To: 1000 ;tag=cUcc2K9Z5Ut1F
Call-ID: 1-1769@10.98.7.36
CSeq: 1 ACK
Contact: sip:sipp@10.98.7.36:5061
Max-Forwards: 70
Subject: Performance Test
Content-Length: 0

SIPP Sample Scenario

SIPP tests SIP by using scenario files to simulate the flow of a call. Then, using parameters passed on the command line, it will run X load, Y times, with Z packet loss %, etc.  So SIPP is working on both a command line and a flat file.

The fat file, henceforth referred to as Scenario file is basically an XML document.  If you are familiar with XML then the Scenario file will look very familiar.  The scenario file will set the constraints of the test.  It will say “i’m sending a packet like this for the invite… now after that’s sent i’m ready to accept 100 trying….” and so forth. 

There are variety of built in scenarios and of course, you can make your own. 

Here’s what a Scenario File looks like:


 
 
 
          Via: SIP/2.0/[transport] [local_ip]:[local_port];branch=[branch]
      From: sipp ;tag=[pid]SIPpTag00[call_number]
      To: [service]
      Call-ID: [call_id]
      CSeq: 1 INVITE
      Contact: sip:sipp@[local_ip]:[local_port]
      Max-Forwards: 70
      Subject: Performance Test
      Content-Type: application/sdp
      Content-Length: [len]       v=0
      o=user1 53655765 2353687637 IN IP[local_ip_type] [local_ip]
      s=-
      c=IN IP[media_ip_type] [media_ip]
      t=0 0
      m=audio [media_port] RTP/AVP 0
      a=rtpmap:0 PCMU/8000     ]]>
 

          optional=”true” start_rtd=”Performance” >
 

 
 

 
 

 
 
 
 
 

 
 
 
          Via: SIP/2.0/[transport] [local_ip]:[local_port];branch=[branch]
      From: sipp ;tag=[pid]SIPpTag00[call_number]
      To: [service] [peer_tag_param]
      Call-ID: [call_id]
      CSeq: 1 ACK
      Contact: sip:sipp@[local_ip]:[local_port]
      Max-Forwards: 70
      Subject: Performance Test
      Content-Length: 0     ]]>
 

 
 
 

 
 
          Via: SIP/2.0/[transport] [local_ip]:[local_port];branch=[branch]
      From: sipp ;tag=[pid]SIPpTag00[call_number]
      To: [service] [peer_tag_param]
      Call-ID: [call_id]
      CSeq: 2 BYE
      Contact: sip:sipp@[local_ip]:[local_port]
      Max-Forwards: 70
      Subject: Performance Test
      Content-Length: 0     ]]>
 

 
 

 
 

 
 

Breaking Down The Scenario

In the above example, there’s a flow going on.  It’s easier to read in an IDE like  Intellij.  The Invite block looks like this:
 
          Via: SIP/2.0/[transport] [local_ip]:[local_port];branch=[branch]
      From: sipp ;tag=[pid]SIPpTag00[call_number]
      To: [service]
      Call-ID: [call_id]
      CSeq: 1 INVITE
      Contact: sip:sipp@[local_ip]:[local_port]
      Max-Forwards: 70
      Subject: Performance Test
      Content-Type: application/sdp
      Content-Length: [len]       v=0
      o=user1 53655765 2353687637 IN IP[local_ip_type] [local_ip]
      s=-
      c=IN IP[media_ip_type] [media_ip]
      t=0 0
      m=audio [media_port] RTP/AVP 0
      a=rtpmap:0 PCMU/8000     ]]>
 

It’s very similar to our real packet from way up above in this post.  There are items in this though in square brackets.  Those items are variables. These are either auto generated or set via the command line parameters.  by default the call_id is auto generated, but some of the other items need to be supplied as a parameter via the command line (such as the service, proxy ip, etc.)

After this invite block we see some xml nodes called recv response.  These allow the test to receive these responses back.
          optional=”true” start_rtd=”Performance” >
 

 
 

 
 

Say for example we didn’t have recv response=”100″… and we send an invite.  If we get a 100 response back, the test will fail. It will say “expecting 180 (the next on the test) but got a 100 instead.

I also have a tag there called “start_rtd”  this starts my response time timer. In this test, I’m organizing the response time between the first 100 I get after an Invite is sent, and the 200 OK.  So it’s saying, I’ve sent an Invite, how long does it take to get a 200 OK? 

Further down we have another packet set up for ACK and then for BYE (disconnect.) 

Built In Scenarios

SIPP comes with some built in scenarios… for example, if you download and install sipp, you could go to the sipp directory and do a ./sipp -sn uac [ip of  your proxy] and it start generating sip load on your proxy based on that scenario.

Notice the -sn parameter with the value uac.  The -sn means use a default scenario.  The value is the name of it. it’s built it. it’s not a flat file. However, if you want to see what uac does, just do this:

./sipp -sd uac   (that will print to screen)
./sipp -sd uac >> uac.xml (that will print it out to an xml doc)

SIPP has several built in scenarios:

  • UAS
  • UAC
  • UAC_Pcap

UAS is a server, it would listen for incoming calls…
UAC is a client sending SIP calls.
In truth, all SIP calls have each party being both the Server and the Client.

UAC_PCAP is a special test, it will send a PCAP (more on this later) to the recipient.  A PCAP is a packet capture.  In this case it’s audio.  In other words, they made a sample audio call, and recorded the packets that created the Audio channel in RTP (RTP is a network layer protocol that handles media streaming.)

Be advised, I was not able to get the built in scenarios to fully 100% work with my PBX here.  My PBX would send some extra calls or expect some extra calls and the tests had to be modified.  You’ll find out what you really need, by making a call from your softphone to the pbx, and recording the traffic with Wireshark and reviewing the results… I’ll detail that in a different blog post.

Using Your Own Scenarios

As mentioned above, you’ll no doubt need to use your own scenarios.  When you modify theirs or create your own, you call it from the command line with -sf [path to your file]

Organizing the Tests

As you can get an idea, these tests are in two parts – Scenario and Parameters.  Because of this, it gets challenging organizing and keeping the actual test.  You can easily store the scenario flat files, but how do you store all your parameters that drive the tests?

I opted for using Jenkins.  Jenkins is a great way to save a test, as a “project” and “build” it on demand.  The command lines are stored in the shell script for the project, and the command line calls the scenario file.  I have more on that in my previous blog post. 

Parameters 

SIPP is half scenario and half parameter driven.  The parameters are supplied at the time of the test, via the command line inputs.  I’ve mentioned some previously to call your own scenario files or use the built in ones… there are tons of parameters you can set… the most common are:
The Service Name (the To: you are sending to)
The Proxy IP of the PBX
Username/pass to authenticate on the PBX

I also use the reporting parameters.  These are all prepended with “trace” like -trace_err (outputs error logs), -trace_screen outputs the general screen stats that displayed and -trace_rtt outputs the response time values to a CSV file (you have to have set a start and stop timer for response time in the test for this to work.)

All these and more can be seen in the documentation on the SIPP repository.

UDP Vs. TCP

This wasn’t a surprise to me, but for some I’ve talked to, it is a bit surprising.  In the web world, mostly TCP is used. But with rapid data transfer, such as an Online Game (Planetside 2, my favorite online game), or VOIP, UDP is used.  UDP doesn’t have any delays checking and needing all the packets delivered. If TCP is used, the phone call would be broken and waiting for lost packets. Packets are going to get lost, so what VOIP does (from my limited vision into VOIP) is use UDP and smooth out the call when packets get lost.  There is  a threshold at which lost packets create a loss in audio. 

Which is why there is a need for load testing. It’s very important to Load test the SIP services and make sure that they can handle the expected load and that the packet loss is within the SLA. 

4 thoughts on “SIP Testing Basics”

  1. Hi,
    We can generate sip loads using SIPP. can we generate only outbound traffic with sipp. Could you pleas give your adivise on to test the incoming calls using sipp?

    1. Hi there, I run single inbound calls with SIPP. the following info will give you a lot of detail but keep in mind if you run load, you need to make sure that you own all the channels so you don’t overload a real carrier.

      Outbound, as you mentioned, is straight forward. For inbound tests, you are doing both an outbound + an inbound call. At my job, a team of people set up phone numbers and give me numbers to test for this inbound purpose. Here’s the set up:

      On my test VM I have a FreeSWITCH running. This FreeSWITCH has a dial plan that takes in a fake DNIS (a DNIS we won’t use in the wild… like 6665554422.

      Now, for part 2, that other team that gives me these numbers… They configure an outbound number to forward back to me.

      Let’s say I want to test Verizon Inbound calls. I get a number that’s a Verizon owned number for testing purposes. We’ll say it’s 8004005555 (just made that up for this example.) I use SIPP to dial 8004005555, which goes out some other carrier… but then ultimately will hit that rule and froward to 6665554422@[My VM with Freeswitch installed]

      So now that SIPP call is returning to me via a specific carrier as an inbound call. My test Freeswitch accepts it, and we handle the communication (perhaps just playing some audio and hanging up.)

      In this scenario, we need to use a reliable outbound call. I use redundancies, so if the first outbound call fails, I try another route.

      Does that help?

  2. Hi,

    Thank you very much for your detailed guidance. As per the SIPP documentation the SIPP can generate the loads or initiate SIP calls. Could you please give your advise on the possibility of the testing on the receiver end which receives multiple SIP calls without initiating SIP calls with SIPP?

    1. Sure. So you can use SIPP in UAS mode as the receiver. It would take the call and complete it. I did that for awhile but I found it had issues with regular repeat calls and tests. So I ended up using a FreeSWITCH install. This way the FreeSWITCH is installed on a VM someplace. Then it has it’s dial plan updated to take a fake dnis like 99955544. My sipp sender sends to 99955444@[my vm that run’s freeswitch’s ip] and freeswitch completes the call. For me, I have it play audio back. But either way the call should complete.

Leave a Reply

Your email address will not be published. Required fields are marked *