Saturday, April 25, 2009

MediaProxy 2.3.x & OpenSIPS 1.5.x Integration

MediaProxy is made up of (at least) two components: A dispatcher and a relay. MediaProxy Server allows specialized processing of clients behind NAT. It has following characteristics:
  • Use of the DNS SRV records to load balance the requests.
  • Can be executed in a seperate server offloading the SIP Proxy.
  • Allows web monitoring.

A detailed description of features and documentation is available at their official site www.ag-projects.com

Latest version can be downloaded at following link:
MediaProxy Download

Basic Steps in Installation:
For people running Debian testing or unstable on an i386 architecture there is an official public repository provided by AG Projects. To use it, add these lines in /etc/apt/sources.list

# AG Projects software
deb http://ag-projects.com/debian unstable main
deb-src http://ag-projects.com/debian unstable main


*** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***

The MediaProxy source already includes the necessary files to build Debian packages. They should probably also work without changes for Ubuntu, though they have not been tested with it.

To build Debian/Ubuntu packages, you can do the following (this is known to work with Debian testing and unstable and should work without changes in Ubuntu 8.04 Hardy as well, though they were not tested there):

apt-get update
apt-get install devscripts cdbs debhelper python-all-dev python-support \
libnetfilter-conntrack-dev python-application python-cjson \
python-gnutls python-twisted-core python-twisted-names \
python-zopeinterface python-pyrad python-sqlobject

Then unpack MediaProxy and in the directory where it was unpacked run:

debuild

You can safely ignore the pgp signing error at the end of the build process, that is only because you do not have the pgp key for the person who is listed as maintainer for the package. The packages are build fine even if they are not signed.

After building them, you can find the .deb packages in the parent directory, from where you can install them using dpkg:

cd ../
dpkg -i mediaproxy-*.deb

or you can install just the ones you need on that particular system. Please note that mediaproxy-dispatcher and mediaproxy-relay both depend on mediaproxy-common so you have to install it too along with either of them.

*** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***


Install the AG Projects debian software signing key:
wget http://download.ag-projects.com/agp-debian-gpg.key
apt-key add agp-debian-gpg.key

Now Run
aptitude update
aptitude install mediaproxy-dispatcher mediaproxy-relay mediaproxy-web-sessions

Log:
root@:/usr/local/src# wget http://download.ag-projects.com/agp-debian-gpg.key
root@:/usr/local/src# apt-key add agp-debian-gpg.key
root@:/usr/local/src# aptitude update
root@:/usr/local/src# aptitude install mediaproxy-dispatcher mediaproxy-relay mediaproxy-web-sessions

Above process complains about dependencies thus follow process below...

Manual Installation Process of MediaProxy:
Prerequisites
~~~~~~~~~
In order to build and install, MediaProxy has the following requirements:

- Linux (at least 2.6.18) with the following features compiled in:
- netfilter support
- connection tracking support
- connection tracking netlink interface
- connection tracking event notification API
- netfilter "NOTRACK" target support
- netfilter "CONNMARK" target support
- netfilter "connmark" match support
- IPv4 connection tracking support
- IP tables support
- IP tables Full NAT support

Distribution provided kernel images should normally provide of all these features as modules. The Debian kernel images have all these features available and can be used out of the box.

- libnetfilter-conntrack (at least version 0.0.89)
Most of the Linux distributions separate a library package into runtime and development packages. To build MediaProxy, the development version is needed (it usually has a -dev suffix in the package name).

- Python (at least 2.4)
http://python.org

- Twisted framework (at least 2.5.0 with epollreactor support)
http://twistedmatrix.com

- python-zopeinterface (this is also a requirement for twisted)
http://zope.org/Products/ZopeInterface

- python-application (at least 1.0.9)
http://pypi.python.org/pypi/python-application

- GNU-TLS
http://www.gnu.org/software/gnutls

- python-gnutls
http://pypi.python.org/pypi/python-gnutls

- python-cjson
http://pypi.python.org/pypi/python-cjson

For the database accounting module:
SQLObject
http://sqlobject.org

For the RADIUS accounting module:
pyrad (at least 1.1)
http://www.wiggy.net/code/pyrad/

You have to install the following software in order to get the conntrack-tools working, make sure that you have installed them correctly before going forward:

  • Linux kernel version >= 2.6.18 (http://www.kernel.org) that, at least, has support for:
- connection tracking system
CONFIG_NF_CONNTRACK=m
CONFIG_NF_CONNTRACK_IPV4=m

- nfnetlink
CONFIG_NETFILTER_NETLINK=m

- ctnetlink (nf_conntrack_netlink)
CONFIG_NF_CT_NETLINK=m

- connection tracking event notification API
CONFIG_NF_CONNTRACK_EVENTS=y

(To check that the event API is enabled in the kernel, make sure you have loaded nf_netlink_conntrack module, run conntrack -E and generate traffic, you should see network events)

  • libnfnetlink: the netfilter netlink library available at:
http://www.netfilter.org/projects/libnfnetlink/files/

  • libnetfilter_conntrack: the netfilter conntrack library available at:
http://www.netfilter.org/projects/libnetfilter_conntrack/files/


Now proceed by first checking the kernal version:
$ uname -a

My kernel version meets the requirement thus i will skip the steps related to kernel install...

Next download files from official site:
http://www.netfilter.org/projects/libnfnetlink/files/
[x] libnfnetlink-0.0.41.tar.bz2 06-Mar-2009 17:31 234K
[x] libnfnetlink-0.0.41.tar.bz2.sig 06-Mar-2009 17:31 65

# tar xvfj libnfnetlink-0.0.41.tar.bz2
# cd libnfnetlink-0.0.41
# ./configure
# make
# make install

Now download:
http://www.netfilter.org/projects/libnetfilter_conntrack/downloads.html#libnetfilter_conntrack-0.0.99

[x] libnetfilter_conntrack-0.0.99.tar.bz2 : GPG signature (key) : md5sum 960c3d347d7f4e3fe7437aa198f36e6e

# tar xvfj libnetfilter_conntrack-0.0.99.tar.bz2
# cd libnetfilter_conntrack-0.0.99

To compile and install conntrack-tools
just follow the classical steps:


# ./configure
# make
# make install

Next check other dependencies:
Download Python 3.0.1 from the link
http://www.python.org/download/ and install as follows:

root:/usr/local/src# tar -xzvf Python-3.0.1.tgz
root:/usr/local/src# cd Python-3.0.1
root:/usr/local/src/Python-3.0.1# ./configure
root:/usr/local/src/Python-3.0.1# make
...
...
Python build finished, but the necessary bits to build these modules were not found:
_curses _curses_panel _sqlite3
_tkinter bz2 readline
To find the necessary bits, look in setup.py in detect_modules() for the module's name.

root:/usr/local/src/Python-3.0.1# make install
...
...
* Note: not installed as 'python'.
* Use 'make fullinstall' to install as 'python'.
* However, 'make fullinstall' is discouraged,
* as it will clobber your Python 2.x installation.

Next download install Twisted Framework 8.2.0 from this link http://twistedmatrix.com/trac/

root:/usr/local/src# tar xvfj Twisted-8.2.0.tar.bz2
root:/usr/local/src# cd Twisted-8.2.0

Download Zope Interfaces 3.1.0 or above:
http://zope.org/Products/ZopeInterface

Untar file and install as follows:
root:/usr/local/src/Twisted-8.2.0# tar xvfz zope.interface-3.3.0.tar.gz
root:/usr/local/src/Twisted-8.2.0# cd zope.interface-3.3.0
root:/usr/local/src/Twisted-8.2.0/zope.interface-3.3.0# python setup.py build
root:/usr/local/src/Twisted-8.2.0/zope.interface-3.3.0# python setup.py install
root:/usr/local/src/Twisted-8.2.0/zope.interface-3.3.0# cd ..
root:/usr/local/src/Twisted-8.2.0# python setup.py install
...
...
Installed /usr/lib/python2.5/site-packages/Twisted-8.2.0-py2.5-linux-x86_64.egg
Processing dependencies for Twisted==8.2.0
Searching for zope.interface==3.3.1
Best match: zope.interface 3.3.1
Removing zope.interface 3.3.0 from easy-install.pth file
Adding zope.interface 3.3.1 to easy-install.pth file

Using /usr/lib/python2.5/site-packages
Finished processing dependencies for Twisted==8.2.0

Above installations also took care of python-zopeinterface, python-application, python-cjson, and GNU-TLS (python-gnutls) dependencies...

Next Database Dependencies:
SQLObject is a popular Object Relational Manager for providing an object interface to your database, with tables as classes, rows as instances, and columns as attributes.

SQLObject includes a Python-object-based query language that makes SQL more abstract, and provides substantial database independence for applications.
root:/usr/local/src# easy_install -U SQLObject

Next download and install pyrad:
pyrad is an implementation of a RADIUS client as described in RFC2865. It takes care of all the details like building RADIUS packets, sending them and decoding responses.

root:/usr/local/src/Python-3.0.1# mv /home/wiz/Documents/pyrad-1.1.tar.gz .
root:/usr/local/src/Python-3.0.1# tar xvfz pyrad-1.1.tar.gz
root:/usr/local/src/Python-3.0.1# cd pyrad-1.1
root:/usr/local/src/Python-3.0.1/pyrad-1.1# python setup.py install
...
...


Installing MediaProxy:
Add DNS SRV records in the for the SIP domains you wish to serve. Put them at the same priority with different weights to allow load balancing of traffic. Put them at different priorities to allow fallback.

Example:

Considering your domain is mydomain.com and you wan to dedicate 3 main servers and 2 fallbacks. The main servers you want to distribute traffic among are 2 1Ghz machines (named nat1 and nat2) and a 2Ghz machine (nat3). You want 25% from the traffic to go to each 1Ghz machine and 50% to the 2Ghz machine. For the fallbacks (nat4 and nat5) you want to distribute the traffic equally among the 2 machines.

_mediaproxy._tcp.mydomain.com. IN SRV 0 25 25060 nat1.mydomain.com.
_mediaproxy._tcp.mydomain.com. IN SRV 0 25 25060 nat2.mydomain.com.
_mediaproxy._tcp.mydomain.com. IN SRV 0 50 25060 nat3.mydomain.com.
_mediaproxy._tcp.mydomain.com. IN SRV 10 50 25060 nat4.mydomain.com.
_mediaproxy._tcp.mydomain.com. IN SRV 10 50 25060 nat5.mydomain.com.

If your DNS provider does not support SRV records, take a look at Managed DNS http://www.managed-dns.org

In this setup, the traffic will be distributed between the nat1, nat2 and nat3 machines in a proportion of 25%+25%+50%. If one goes offline the other 2 will take it's traffic share redistributing the load balancing according to their own weight. If all of nat1, nat2, nat3 stop responding, nat4 and nat5 will be used distributing the traffic among them equally (50%+50%)

# mkdir /etc/mediaproxy
# mkdir /etc/mediaproxy/tls
# cd usr/local/src/mediaproxy

Download and copy in the directory:
# tar xvfz mediaproxy-2.3.4.tar.gz
# cd mediaproxy-2.3.4
# ./setup.py build
# ./setup.py install

running install
running build
running build_py
running build_ext
running build_scripts
running install_lib
running install_scripts
changing mode of /usr/bin/media-dispatcher to 755
changing mode of /usr/bin/media-relay to 755
running install_egg_info
Removing /usr/lib/python2.5/site-packages/mediaproxy-2.3.4.egg-info
Writing /usr/lib/python2.5/site-packages/mediaproxy-2.3.4.egg-info

Enable IP Forwarding by making changes in /etc/sysctl.conf file...
Permanently enable ip forwarding,,,
# gedit /etc/sysctl.conf &
...

# Uncomment the next line to enable packet forwarding for IPv4
net.ipv4.ip_forward=1

# echo 1 > /proc/sys/net/ipv4/ip_forward
# sysctl net.ipv4.ip_forward
net.ipv4.ip_forward = 1


+ Make directory, copy necessary files to proper directories, and setup ownership:
$ mkdir /var/run/mediaproxy
$ cp config.ini.sample /etc/mediaproxy/config.ini
$ cp tls/* /etc/mediaproxy/tls/

Starting MediaProxy Dispatcher...
# ./media-dispatcher --no-fork
# ./media-dispatcher &


OpenSIPS+MediaProxy Integration:
# mkdir /var/run/mediaproxy
# gedit config.ini &

Make following changes in file...

[Relay]
dispatchers = mydomain.net:5060 _sip._udp.mydomain.net
port_range = 50000:60000
log_level = DEBUG
on_hold_timeout = 1800

[Dispatcher]
listen = 10.1.10.205:50000
management_use_tls = no
log_level = DEBUG
accounting = radius

[Radius]
; This section needs to be configured if radius accounting is enabled
; OpenSIPS RADIUS configuration file. All RADIUS cofiguration parameters will
; be read from this file, including dictionary files.
;
;config_file = /etc/opensips/radius/client.conf
config_file = /etc/freeradius/clients.conf

; Additional dictionary file with MediaProxy specific attributes.
;additional_dictionary = radius/dictionary
additional_dictionary = /etc/freeradius/dictionary

[OpenSIPS]
socket_path = '/var/run/opensips/socket'


Now edit opensips.cfg file to include MediaProxy...
#
# ----- nat_traversal
#**
loadmodule "nat_traversal.so"
#**
modparam("nat_traversal", "keepalive_interval", 90)
modparam("nat_traversal", "keepalive_method", "OPTIONS")
modparam("nat_traversal", "keepalive_from", "sip:keepalive@mydomain.com")


#
# --> nathelper
#**
loadmodule "nathelper.so"
#**
modparam("nathelper", "natping_interval", 60)
modparam("nathelper", "ping_nated_only", 1)
modparam("nathelper", "sipping_bflag", 7)
modparam("nathelper", "received_avp", "$avp(i:801)")
modparam("nathelper", "sipping_from", "sip:pinger@mydomain.com")

#
#**
loadmodule "mediaproxy.so"
#**
#--------------- Default Values
modparam("mediaproxy", "mediaproxy_socket", "/var/run/mediaproxy/dispatcher.sock")
modparam("mediaproxy", "mediaproxy_timeout", 500)
modparam("mediaproxy", "signaling_ip_avp", "$avp(s:nat_ip)")
modparam("mediaproxy", "media_relay_avp", "$avp(s:media_relay)")

#--------------- Main Routing Logic

#

17 comments:

ayed said...

good article, but the point in opensips with mediaproxy where most looking for a solution is the file opensips.cfg which in your case you just show few lines of it, so what is the point you want to make, the configuration file is the most important part not how to install mediaproxy.

Khan said...

Ayed,
Sorry about the late reply, i was on vacation just got back and in process of getting my feet wet again. I agree with you, but I didn't quiet finished and had success in running everything smoothly but trust me when i do, you will see the configuration file too :)

ayed said...

hi
thank you, the reason why I said so, is a lot of articals and wiki about opensips and you will never find a simple example in the inernet of how you use it with mediaproxy or rtpproxy, the developer themselves they don't do that in opensips website and claim that is free and is GPL, there were before sipwizar but is only for openser 1.2. Hope that the developer of opensips can notice that and stop releasing new version without giving any examples of how to use them

Khan said...

Ayed, I'm trying to get one thing functional and working, unfortunately I am doing this as part time. I agree with you 100% because I am in the same shoes and tired of searching for config file. I read a book by packet publishing but that is outdated too, since it covers older version and nothing like new version. Just keep your finger crossed, and i will post the working config without a second thought as soon i have finished testing all this. In the mean time, if you have specific question please ask, i will try to answer to the best of my knowledge or i will point you in right direction :)

Amit Aggarwal said...

I am having hard time to solve NAT problem in my network. I tried both RTPproxy and Mediaproxy. But I think lack of documentation is the reason. Can you tell what kind of problem you are facing and can you share your opensips.cfg with me?

Khan said...

Amit,
I'm using RTPproxy and having problem with rtp packets being lost, cfg file is not functional since i only get one way audio, if you like i can pass it to you may be you can see something i can not. I will put it in here as soon as i make it functional. I do not have any problem sharing anyway :)

Amit Aggarwal said...

Khan,

Please send me the config file. I am getting sound in either direction may i am able to find something and that we can help each other. Thanks!

Khan said...

Amit,
I have included CFG file in my RTPproxy howto. Please view the blog related to RTP. Also if you have any idea what might be the case leave comments and i will try...

Khan

dhany said...

nice article khan,,
could you help me to explain in SER (SIP Express Router) about mediaproxy configuration.
Because, I think in this file of mediaproxy directory use for opensips,not for SER, it is my assume,,
thanks for help,,

dhany

Khan said...

dhany,
I never worked on SER thus I do not know how to implement mediaproxy on that, I believe its IPTEL product, but later on it converted to OpenSIPS, i might be wrong here. If you have specific question please shoot, i will try my best to answer or do research on it :)

dzakone said...
This comment has been removed by the author.
dhany said...

thankq khan,,
btw,i think the previous version of mediaproxy 2.3 is compatible with SER and mediaproxy module in version 2.3 divide become media-relay and media-dispatcher.
In previous version,mediaproxy module only mediaproxy (can assume media-relay and media dispatcher).
This is my opinion when i compare mediaproxy 2.3 with previous version.
Do you agree?
CMIIW,,
:)

Johan said...

I ran into a dependency problem, is ther any way to get around it.

Further is there maybe a VMWARE imaga around that has the right kernel?


debian:~# aptitude install mediaproxy-dispatcher mediaproxy-relay mediaproxy-web-sessions
Reading package lists... Done
Building dependency tree
Reading state information... Done
Reading extended state information
Initializing package states... Done
Reading task descriptions... Done
The following packages are BROKEN:
mediaproxy-common python-application python-gnutls
The following NEW packages will be installed:
mediaproxy-dispatcher mediaproxy-relay python-cjson{a} python-dns{a}
python-elementtree{a} python-formencode{a} python-pkg-resources{a}
python-pyrad{a} python-sqlobject{a} python-twisted-names{a}
0 packages upgraded, 13 newly installed, 0 to remove and 28 not upgraded.
Need to get 1066kB of archives. After unpacking 6005kB will be used.
The following packages have unmet dependencies:
mediaproxy-common: Depends: python-support (>= 0.90.0) but 0.8.4lenny1 is installed.
Depends: libnetfilter-conntrack3 (>= 0.0.100) which is a virtual package.
python-gnutls: Depends: python-support (>= 0.90.0) but 0.8.4lenny1 is installed.
python-application: Depends: python-support (>= 0.90.0) but 0.8.4lenny1 is installed.
The following actions will resolve these dependencies:


Greetings from Germany

Khan said...

Did you try updating ($sudo aptitude update) seems like your Python is broken, see if you can uninstall and reinstall it! You need a little research about libnetfilter-conntrack3 to see what repositories you needed to add in order to update these dependencies successfully. My apologies if i have not been much help since its 1am here and I'm still struggling with somethings :)
How is football going man, who you rooting for :)

Johan said...

Hi, yeah, same for me other work keeps me away from exiting things. With respect to football :-)
"once a Dutch ....."
I will try to fix what you recommended and see if it works then.

Satya said...

When I am running media dispatcher in media proxy, I am getting application error

from application.process import process, ProcessError
ImportError: No module named application.process

Satya said...

How do i uninstall mediaproxy ? Or How do i upgrade to latest version of mediproxy? I need the uninstallation steps.