An SMS reply back solution can be used to automate ticketing, voting, polls, opt-in and out, among many other use cases. A mobile subscriber can send an SMS with a certain keyword and receive an automatic reply. Let's build this kind of solution using free software only: the open-source Kannel gateway and the free SMS Proxy app for Android.
SMS Center (SMSC)
The Short Message Service Center (SMSC) is the mobile network element responsible for the SMS messages routing, storage and delivery.1 Usually it can also handle billing, filtering (i.e. spam protection) and other tasks related to SMS management. Anyway, the main purpose of the SMSC is to store and deliver SMS messages. It stores them until the destination subscriber becomes available to receive or the messages expire, whichever comes first. The role of the SMSC is very similar to an e-mail server. The e-mails are not delivered peer-to-peer. They are delivered through an e-mail server based on a store-and-forward model. An SMSC is like an e-mail server for SMS. The mobile operator can have multiple SMSC. When you send an SMS to another user, the SMS will first reach the SMSC. After that, the SMSC will try to dispatch it to the end user.
There are special cases when the SMSC doesn't work in store-and-forward mode or there is no SMSC at all and the SMS are delivered to the end user directly. These special cases are beyond the scope of this article.
Since the SMSC is part of the mobile network, the communication between the mobile and the SMSC is wireless. However, the service providers prefer to have a dedicated connection to the SMSC. This can be more efficient and especially useful to handle large volumes of traffic. The mobile operator may allow them to access the SMSC through a dedicated interface using the SMPP protocol. There are several alternative protocols available: CIMD2, UCP, SEMA and even some HTTP-based APIs, but SMPP is the de facto standard since the 90s. Currently there are more than 30 SMSC vendors and virtually all of them support SMPP. There are also SMS hubs and Bulk SMS providers. Usually they expose an API or dedicated interface to send and receive SMS in the same way as an SMSC.
SMS Gateway
Strictly speaking, it is not mandatory to use an SMS gateway to send and receive SMS. It is possible to interact with the SMSC directly. The SMPP protocol is an open standard.2 Everybody can develop an SMPP client or reuse an open source library like OpenSmpp to interact with the SMSC without any middlewares or proxies. However, using an SMS gateway can be convenient to avoid writing your own implementation. A typical SMS gateway may support different SMSC protocols, not only SMPP. It may support SMS filtering, SMSC connection multiplexing, routing, queuing, logging, user authentication and so on. The open-source Kannel gateway has all these features and many more.
Of course, you can use a different SMS gateway. There are many free alternatives. For example, the Apache Camel has a built-in SMS gateway. Jasmin is another open source SMS gateway. There are many commercial SMS gateways as well. Nonetheless, in terms of stability and performance Kannel is perfect for almost any kind of SMS project. Some mobile operators are using Kannel even for business critical services.
SMSC Proxy
An SMSC proxy is any kind of intermediary service between the SMS client and the SMSC. For example, the SMS gateway can also act as an SMSC proxy. There can be multiple proxies, one behind the other, also known as a proxy chain. Usually an SMSC proxy behaves as a transparent proxy. This means that the proxy acts like a real SMSC and the SMS client needs not to be aware of the proxy presence.
The mobile operators often prefer to provide access to an SMSC proxy instead of the real SMSC. The proxies can provide another layer of protection, load balancing, filtering, access control and so on.
In this article, the SMSC proxy is an Android app. As any other kind of mobile equipment, the typical smartphone is capable of sending and receiving SMS. Hence, it's possible to use an app to turn the smartphone into an SMSC proxy. A mobile can't send or receive a large amount of SMS. Usually the device and the mobile operator limit the throughput to a few SMS per minute. If the limit is exceeded, the operator can block the subscription from sending more SMS temporarily or permanently. That's why, if you need to send thousands of SMS daily, you will definitely need to use a dedicated SMSC connection, not a single mobile phone. The SMSC proxy described in this article is intended to save time and costs when prototyping an SMS service, doing unitary tests or building a low-traffic solution. Even though the app is free, be aware that the mobile operator may charge a fee for every SMS. In some countries, the prices per SMS can be rather high. It is recommended to check the local carrier fees for SMS messaging to avoid unexpected charges.
Setting up the SMS Proxy app
The app can be downloaded from the Amazon appstore. The SMS Proxy Lite version is free for any purpose, including commercial use. It doesn't have any limitations, ads or nags.
If the device has Android 6.0 (Marshmallow) or newer, when running the app for the first time, it will request to grant the permissions to send and view SMS messages. Granting these permissions is necessary to allow the app to send and receive SMS. Older Android versions will not request to grant these permissions explicitly.
The main screen has 2 configuration parameters:
- Port is the TCP port number where the proxy will listen for the inbound SMPP connection. It can be any value in the range from 1024 to 65535. According to IANA,3 the registered port number for SMPP is 2775. In the SMS Proxy app the default value is 5011.
- Limit the traffic to a certain number of SMS per minute. This limit can be any value in the range from 1 to 100. The default value is 10. By default, Android limits the outbound SMS traffic to a maximum of 30 messages every 30 minutes. If you need to increase this limit, try to google for
sms_outgoing_check_max_count. This limit is a safety feature. If you increase it, there is a risk that the outbound SMS traffic may exceed the limits enforced by the mobile carrier. In that case, the subscription can be blocked. Do remember that the operator may charge for every SMS. Therefore, don't try to increase the limits unless really necessary. Neither Android nor the app limit the incoming SMS traffic.
If the port or the SMS limit are modified, the new values are taken into account the next time the SMSC is started. Push the Start button to start the SMSC.
On Android 6.0 (Marshmallow) or newer, there is a battery saving feature which restricts network access after a few minutes of inactivity, if the charger is disconnected. The app will warn you about this feature when the SMSC is started for the first time. In order to prevent the SMSC from being disconnected after a few minutes, you can either keep the battery charging or disable battery optimization for the SMS Proxy app. The latter can be done by pressing Settings. You will see a list of apps and the corresponding battery optimization status. By default, the list will contain only the apps without battery optimization. You can adjust the filter to list all the apps. After that, you can scroll the list to find the SMS Proxy app and disable battery optimization. This should prevent unexpected disconnections. The SMS Proxy app is designed to use as little battery as possible, even when battery optimization is turned off.
If WiFi is active, the proxy will bind to the WiFi adapter IP address. The status string will report that the SMSC is waiting for connection on X.X.X.X:YYYY, where X.X.X.X is the IP (typically 192.168.X.X) and YYYY is the port number configured above.
If WiFi is not active, the IP address will not be shown. In this case, it is possible to connect the SMS client through USB by configuring port forwarding with ADB, for example:
adb forward tcp:5011 tcp:5011
While the ADB tunnel is running, the proxy will be accessible at 127.0.0.1:5011, unless you changed the port number.
Take note of the IP and port number. These values will be configured in the Kannel gateway to establish the SMPP connection. Keep the SMS Proxy app running in the foreground to avoid disconnections.
Setting up Kannel
Kannel is mainly used on Linux systems. Precompiled binaries are available for many popular Linux distributions. Usually, installing Kannel on Linux takes a single click or a few shell commands. It is possible to make It run on Windows and other OS. If you have a good reason not to use Linux—although, I can hardly imagine one—there are multiple tutorials available explaining how to run Kannel with Cygwin for Windows. In this article we are not going to use any external tools, but in a real life SMS project it is always necessary to implement some scripts and webservices to customize and automate the Kannel operation. Linux and other Unix OS have all the necessary tools available out of the box. That could be one of the reasons why most of the large scale Kannel deployments are done on Linux.
If you are new to Linux or you don't want to spend a long time installing Linux, you can download a ready to use Linux image for VMware or VirtualBox. Several websites provide this kind of images for free, for example: OSboxes. When using a virtual machine to run Linux, make sure to bridge the network adapters to provide connectivity between Linux and your Android phone.
My test server is CentOS 7 for x64. Therefore, I picked up this installer packaged by Roman Dmitriev: kannel-1.4.4-1cnt7.x86_64.rpm. After downloading the RPM, it can be installed directly, without any prerequisites:
wget 'ftp://***/kannel-NNN.rpm' sudo rpm -ivh kannel-NNN.rpm
Note: In the above example, just replace the *** and NNN with the appropriate URL and filename, depending on the Linux distribution and CPU.
Before starting up Kannel, let's setup its main configuration file. The default location is /etc/kannel.conf:
group = core admin-interface = 127.0.0.1 admin-port = 13000 admin-password = change-me smsbox-interface = 127.0.0.1 smsbox-port = 13001 sms-resend-retry = 1 access-log = /var/log/kannel/access.log log-file = /var/log/kannel/kannel.log log-level = 1 group = smsbox bearerbox-host = 127.0.0.1 sendsms-interface = 127.0.0.1 sendsms-port = 13013 global-sender = 123 group = sendsms-user username = "" password = "" default-smsc = celersms # This is our connection to the SMS Proxy app # 192.168.X.X is the smartphone WiFi adapter IP group = smsc smsc-id = celersms smsc = smpp host = 192.168.20.22 port = 5011 transceiver-mode = 1 smsc-username = "" smsc-password = "" system-type = "" log-file = /var/log/kannel/smpp.log # Catch keywords YES and NO (case insensitive) group = sms-service keyword-regex = ^(?i)(yes|no)$ catch-all = 1 text = Message received (%t) max-messages = 1 # Catch all other SMS and ignore them group = sms-service keyword = default text = N/A max-messages = 0
Under the core section, we need to setup the admin interface. This is mandatory, even if we are not going to send any admin commands. For security reasons it is recommended to avoid binding the Kannel ports to public network interfaces, unless necessary. In this example, we are binding all ports to the loopback interface 127.0.0.1 to restrict remote access. If you do need to allow remote access, make sure to enable SSL and configure some strong passwords to minimize the security risks.
Our Kannel gateway will open the following three ports:
- 127.0.0.1:13000 is the mandatory admin interface
- 127.0.0.1:13001 is the SMS box service
- 127.0.0.1:13013 is the sendsms port
If any of these ports are already in use, feel free to change them. Take note of the sendsms port because it will be used to send some test SMS messages.
Notice that there are two SMS services defined. The first one uses a regular expression to capture all messages with keywords “YES” and “NO” (case insensitive) and replies with a timestamp when the message was received. This service is our reply back SMS. The second service captures all other SMS and ignores them. It is necessary to configure a default service to ignore unwanted SMS because otherwise Kannel will reply with an error message.
All logs are saved to /var/log/kannel:
- kannel.log is the main Kannel gateway log. It's the first log to check if the gateway fails to start.
- access.log records the SMS. If you need to generate the list of subscribers who sent a specific keyword during a specific period of time, the access log provides this information.
- smpp.log has the SMPP traces. If there is an error trying to connect to the SMS Proxy app, this log can have more details about the issue.
Sending and receiving SMS
Let's try to start the gateway. It can be done using the service command:
sudo service kannel start
Check kannel.log to see if there is any error. If the gateway started up fine, check the smpp.log to see if the connection to the SMS Proxy app was established successfully. This can be seen in the app UI as well. If the connection is failing, double-check that the app is running in the foreground, it is not affected by the battery saving settings, the IP and port are correctly configured and the connectivity is not blocked by a firewall.
If everything is fine, let's try to send a test SMS (replace 1234567 with the destination number):
wget 'http://127.0.0.1:13013/cgi-bin/sendsms?username=&password=&to=1234567&text=Hello'
Usually it is possible to send the SMS to yourself by specifying your mobile number as the destination number. This can be useful if you don't have another mobile at hand. In this example, we are not specifying a username and password to send the SMS because we created an empty username in kannel.conf. Bypassing the authentication is not recommended for a real life production service.
Let's try to send a Unicode SMS with some emoji. Notice that we added the parameter coding=2 to specify Unicode encoding:
wget 'http://127.0.0.1:13013/cgi-bin/sendsms?username=&password=&to=1234567&coding=2&text=%D8%3D%DE%00'
The hexadecimal value for the emoticons and emoji can be found in this table (refer to the UTF16 column). For more information on SMS encoding with emoticons and emoji, refer to this article.
If sending SMS is working fine, the next thing to test is the reply back service. This service will trigger a response when the incoming SMS has the keyword “YES” or “NO”. A typical real life SMS reply back invokes a webservice URL to process the matching SMS. For example:
group = sms-service keyword-regex = ^(?i)(yes|no)$ catch-all = 1 get-url = "http://127.0.0.1:13013/cgi-bin/sendsms?username=&password=&to=%p&text=Message+received+(%t)" max-messages = 0
The above configuration is using the sendsms API to send the response. The result will be the same as before. However, you can configure your own webservice to process these messages and implement more complex and customized scenarios. Notice that in this case max-messages is set to zero to avoid responding twice.
Kannel supports many advanced features and different types of messages. All these use cases are covered in the official documentation. Additionally, the SMS Proxy Pro app can be used to send binary SMS, if the Android device supports this feature. Binary SMS can be used to perform remote device configuration, push interactive content, track the mobile location and much more.
References
- «Technical realization of the Short Message Service (Release 16)», ETSI (2020)
- «Short Message Peer-to-Peer Protocol Specification Version 5.0», SMS Forum (2003)
- «Service Name and Transport Protocol Port Number Registry», Internet Assigned Numbers Authority
