View source for Regular expressions


== About regexroute module ==

The regexroute module provides a simple way of routing telephony calls inside Yate. This module describes the routes using a configuration file in which each number is matched using regular expressions.

For more informations about how regular expressions in [[What_is_Yate|Yate]] are working use "man grep" or you may use a graphical interface like http://laurent.riesterer.free.fr/regexp/

There are two messages handled by the regexroute module:
* the "preroute" message that tries to classify the caller in a context
* the "route" message that tries to map a called number to a string describing the target channel or module

In the preroute stage the caller is matched with regular expressions in the [contexts] section and the result is stored in a message parameter named "context". Note that the preroute stage is skipped if the message already has a "context" parameter.

In the route stage the called number is matched against regular expressions from the section with the same name as the value of the "context" parameter. If this parameter is missing the "default" section is used.

In both prerouting and routing the result of the match (right-hand size of the line, after the equal sign) can include parts of the matched string. For this to work the matching expression should include the \( and \) subexpression capture markers. In the result you can use \digit markers which are replaced with the entire matched string (\0), first captured subexpression (\1), second matched subexpression (\2) and so on. You can include each of the the \digit placeholders as many times as you need

Expressions are checked sequentially in their section. As soon as one matches and specifies a result, its result is returned and no further matches are attempted. If an expression matches, but its result is empty, parameters will be changed according to the string after ";", but further matches will be attempted (see the gsm codec example which doesn't return).

You can act on non-matching regular expressions by adding a caret character ^ at the end of them. Note that in this case the matched string (\0) and all captured subexpressions will be empty.

If no expression matched the corresponding message handler will not consider the message handled, giving an opportunity to route the call in another module. The priority of each handler is controlled by the [priorities] section.

First word of a matched target has a special meaning:
* if - makes a new match using the rest of the line up to the '''=''' character
* return - returns immediately from the context without routing
* include - calls another context, returns at the next entry if the other context did not return successfully
* jump - jumps to another context, does not return to this context
* match - modifies the matched string instead of specifying a target
* rename - changes the name of the message
* enqueue - puts a new message in the engine, parameters are taken from the old message but placed in the new one
* dispatch - dispatches a new message in the engine waiting for it to return, parameters are taken from the old message but placed in the new one
* echo - just displays that line after making substitutions

It is possible to set message parameters by appending them as name=value while separating them with semicolons (;)

The module also supports global variables that are not tied to a specific message. These can be used in the right hand expressions for building call targets and setting message parameters. Global variables are accessed as $(variable) or as $variable in function parameters.


=== Order of processing ===
For each line that matches the following operations are applied:
* line is broken at semicolon ''';''' characters
* matches of the form '''\N''' are replaced
* parameters of the form '''${name}''' are replaced
* functions of the form '''$(name,etc.)''' are evaluated
For the ''echo'' target the line is not broken at semicolons. 
For the ''if'' target the line is processed only after matching the last ''if'' - that is on the first target that is not ''if''.


==== Examples ====

1) For the matching rule: 
^00\(.*\)$=iax/\1@internat.ion.al/\0 
(match strings starting with 00 and capture everything else to the end of string)

And the called number: 
0099123456

The routing will return: 
iax/99123456@internat.ion.al/0099123456

2) For the matching rule: 
^123456$=return;called=johndoe

And the called number: 
123456

The routing will dial the registered user johndoe.

''Just remember to have the route priority numerically lower (which gives it a higher actual priority) than the regfile (that routes the call to johndoe)!''

More in-depth explanation of the sentence above (as I had a hard time understanding it myself :-) ): 

* regexroute.conf: i.e. preroute=10 route=10 
* regfile.conf: i.e. preroute=50 route=50

YATE will start routing with regexroute.conf (highest priority!). There the number ''123456'' matches, changes the called parameter to ''johndoe'' and returns. As the number has not been routed yet, YATE tries the files with the lower routing priority, now trying to route ''johndoe''. ''johndoe'' is found in regfile.conf, and the originally called number ''123456'' is routed to the user ''johndoe''.

Naturally, instead of regfile.conf you can use the database module (register.conf), with the priorities set accordingly. Note, if you have a catch-all in regexroute, and give it a higher numeric priority, then it will route all your internal lines - meaning, you need to define rules in regexroute so these numbers get routed by your register or regfile modules.

=== Testing and debug routing ===

An easy way to debug routing is to use the function echo

^.*$=echo called is ${called}

The first part - ^.*$ matches every call and the second part prints in the Yate Debug called is called_value

=== Handling authentication ===

In many cases you need to authorize only authenticated calls. The authentication is performed by the module that handles the incoming call which sets the "username" parameter. This is valid only for the SIP protocol.

{|
|; reject unauthenticated calls, give challenge-response protocols a second chance:
${username}^$=-;error=noauth
|}

=== Changing codec priorities ===

The incoming call's codecs are hold in a parameter "formats" that can be manipulated to favor some codecs. An example of bringing the GSM codec to front is this:

{|
|; move the GSM codec to front if not already there
 ${formats}^\(.*\),gsm\(.*\)$=;formats=gsm,\1\2
|}

There is no need to match ''gsm'' with no comma in front of it because it would already be first!

See more info about using regexroute for transcoding on a [[Transcoding|separate page]].

=== Request inband DTMF detection ===

If inband DTMF tones are used and the hardware does not support detecting them Yate can provide detection using the [tonedetect] module. Please check the module documentation for applicability and resource format.

The request for attaching a DTMF detector can be specified from routing, where a boolean true value stands equivalent to ''tone/*'' :

{|
|; request DTMF detection on inbound calls from PSTN
 ${id}^sig/=;tonedetect_in=yes
 ; request DTMF detection on all outbound calls - CPU consuming!
 .*=;tonedetect_out=yes
|}


=== Handling extra messages ===

It is possible to use this module to install handlers for arbitrary messages. This is done by listing the messages and their priority in the [extra] section of the config file. You can only install one handler for any given message name.

Each such message will be handled in a section with the same name as the message. The default match sting is the message name so make sure you set a new match or name the parameters explicitely:

{|
|[extra]
; insert an extra authentication handler
user.auth=50

[user.auth]
; blacklist any private 10.0.x.y IP address
${ip_host}^10\.0\.=-;error=forbidden;reason=Your IP range is blacklisted
; don't waste resources querying the database if no credentials are provided
${username}^$=;auth_register=false
|}

=== Using global variables ===
The module supports arbitrary variables that are preserved between calls.

These variables can be initialized when Yate starts by listing them in the [$once] section or they can be reinitialized each time by placing them in the [$init] section. The variables can also be created by assigning them a value.

To access a global variable use ''$(varname)'' or, if in a function, $(funcname,...,''$varname'',...)

To set a global variable use: ...;''$varname''=expression

To delete a global variable use: ...;''$varname''

The following recipe allows retrieving and setting global variables from the [rmanager] command line:

{|
|
 [extra]
 engine.command=150

 [engine.command]
 .*=newmatch ${line}
 ^get \([^ ]\+\)$=\1=$(\1)$(chr,13)$(chr,10)
 ^set \([^ ]\+\) \+\(.*\)$=\1=\2$(chr,13)$(chr,10);$\1=\2
|}

=== The regexroute configuration file ===

{|
; Minimalistic basic regular expressions information
;  ^ matches start of string
;  $ matches end of string
;  . matches any character
;  [list] matches one character in the list
;  [^list] matches one character not in list
;    Lists can be individual characters or ranges char-char. You can insert ]
;    by making it the first character in list and ^ by making it not the first
;    character
;  * matches preceeding expression any number of times (including zero)
;  \+ matches preceeding expression at least one time
;  \? matches preceeding expression zero or one time
;  \{N\} matches preceeding expression exactly N times
;  \{N,\} matches preceeding expression N or more times
;  \{N,M\} matches preceeding expression between N and M times
;  \( \) captures the contained subexpression
; Remember matches are greedy, they will match as much as possible
; You must escape ^ $ . * [ and \ with \ whenever you want them to be normal
;  characters except in lists
; Please see the manual pages for grep and sed for more information

; Functions callable in the right-hand side:
;  $() = a ; character
;  $($) = a $ character
;  $(++N) = N+1
;  $(--N) = N-1
;  $(length,STRING) = length of STRING
;  $(upper,STRING) = STRING converted to upper case
;  $(lower,STRING) = STRING converted to lower case
;  $(chr,N) = character with numeric code N
;  $(hex,N[,LEN]) = little endian hexadecimal value of N with space between octets
;  $(add,VAL1,VAL2[,LEN]) = VAL1+VAL2 left filled to LEN or length of VAL1
;  $(sub,VAL1,VAL2[,LEN]) = VAL1-VAL2 left filled to LEN or length of VAL1
;  $(mul,VAL1,VAL2[,LEN]) = VAL1*VAL2 left filled to LEN or length of VAL1
;  $(div,VAL1,VAL2[,LEN]) = VAL1/VAL2 left filled to LEN or length of VAL1
;  $(mod,VAL1,VAL2[,LEN]) = VAL1%VAL2 left filled to LEN or length of VAL1
;  $(eq,VAL1,VAL2) = "true" if VAL1 = VAL2 (numerically), "false" otherwise
;  $(ne,VAL1,VAL2) = "true" if VAL1 != VAL2 (numerically), "false" otherwise
;  $(lt,VAL1,VAL2) = "true" if VAL1 < VAL2, "false" otherwise
;  $(gt,VAL1,VAL2) = "true" if VAL1 > VAL2, "false" otherwise
;  $(le,VAL1,VAL2) = "true" if VAL1 <= VAL2, "false" otherwise
;  $(ge,VAL1,VAL2) = "true" if VAL1 >= VAL2, "false" otherwise
;  $(streq,VAL1,VAL2) = "true" if VAL1 = VAL2 (string), "false" otherwise
;  $(strne,VAL1,VAL2) = "true" if VAL1 != VAL2 (string), "false" otherwise
;  $(random,STRING) = STRING with each ? character replaced with a random digit
;  $(index,N,ITEM1,ITEM2,...) = N-th (modulo length of list) item in list
;  $(rotate,N,ITEM1,ITEM2,...) = list rotated N (modulo length of list) times
;  $(runid) = the current Engine run identifier
;  $(nodename) = the node name the Engine runs as, may be empty
;  $(threadname) = name of the thread that dispatched the message, may be empty
;  $(dispatching) = the reentry depth, 0 if the message is not generated locally
;  $(transcode,FLAGS,FORMAT1,FORMAT2,...) = list of formats the input can be transcoded into
;       e - exclude initial formats form generated list
;       r - allow rate conversion (for use with wideband)
;       c - allow changing channels number
; Note that functions ++, --, index and rotate will automatically update N
;  if it is a variable in the $varname format.


[priorities]
; Set the priorities for the insertion of the regular expression module in the
;  handler chain; a priority of 0 disables the handler entirely

; preroute: int: Priority of the prerouting message handler
;preroute=100

; route: int: Priority of the routing message handler
;route=100

; extended: bool: Use extended regular expressions
;extended=no

; insensitive: bool: Make the regular expressions case insensitive
;insensitive=no

; prerouteall: bool: Preroute even calls having a context or with empty caller
;prerouteall=no


[$once]
; First-time only global variables initialization.
; It is executed during first initialization before the [$init] section
; Each line must be of the form:
;  varname=value


[$init]
; Reload time global variables initialization
; Each line must be of the form:
;  varname=value


[extra]
; This section allows installing handlers for any message name.
; Each line must be of the form:
;  message.name=priority
; You can only install one handler for any given message name.
; For each handler create a corresponding [message.name] section in which
;  implement handling for that specific message. You will need to match
;  parameters explicitely or set a new match string.


[contexts]
; This section is used by the prerouting handler to classify calls by the
;  caller name; each call is assigned an input context (only if none exists
;  already) that is used later in the routing stage
; Expressions are scanned from top to bottom; the first match returns the value
; Each line must be of the form:
;   regexp=context_name
; To match a message parameter you can use the format:
;   ${paramname}regexp=context_name
; Strings captured with the regular expression construct \(...\) can be
;  inserted in the context name using \1, \2, \3, ... while \0 holds the entire
;  matched regexp even if no capture was used
; Message parameters can be inserted in the context name using ${paramname}
;
; Example:
;^$=empty
;^00=international
;^0=longdistance
;.*=default


[default]
; Sections like this one are used by the routing handler to find the target
;  of calls by the called name
; The [default] context is special, it is used when no context has been set
;  otherwise you have to place the entries in a section with the same name
;  as the context
; Expressions are scanned from top to bottom; the first match returns the value
; Each line must be of the form:
;   regexp=target
; To match a message parameter you can use the format:
;   ${paramname}regexp=target
; To match a function possibly containing parameters you can use the format:
;   $(function,param...)regexp=target
; To act on non-matching expressions add a ^ at end of the regexp. In this
;  case the \0 ... \9 replacements will always be empty
;   regexp^=target
; Strings captured with the regular expression construct \(...\) can be
;  inserted in the target using \1, \2, \3, ...
; Message parameters can be inserted in the target using ${paramname}
; Functions can be inserted using $(function,param,param)
;
; First word of a matched target has a special meaning:
; if - make a new match using the rest of the line up to the = character
; return - returns immediately from the context without routing
; include - calls another context, returns at the next entry if the other
;   context did not return successfully
; jump - jumps another context, does not return to this context
; match - modify the matched string instead of specifying a target
; rename - changes the name of the message
; enqueue - puts a new message in the engine, parameters are taken from the
;   old message but placed in the new one
; dispatch - dispatches a new message in the engine waiting for it to return,
;   parameters are taken from the old message but placed in the new one
; echo - displays that line after making substitutions
;
; It is possible to set message parameters by appending them as name=value
;  while separating them with semicolons (;)
; Placing just the parameter name without the =  sign will clear the parameter
; Using $name=value will instead change the global variable with that name.
; Similarily specifying just $name will clear the global variable
;
; Please note that the match string is not changed together with the message
;  parameter from which it was copied; for example in routing stage using
;  "match 123" and ";called=123" have different effects
;
; Example:
;   route the emergency 112 and 911 numbers to POTS, any channel on an E1,
;   force specific data format too
;^112$=zap/1-31; format=alaw
;^911$=zap/1-31; format=mulaw
;   route international calls over SIP, replace caller name
;^00\(.*\)$=sip/sip:\1@international.gateway ; callername = International call
;   route value added services over IAX, trailing part is sent as IAX extension
;^09\(.*\)$=iax/vap@gateway.for.vap/\1
;   route green calls over IAX with 2 digits used to form an user name,
;   remaining digits are sent as extension
;^08\(..\)\(.*\)$=iax/green-\1@gateway.for.green/\2
;   everything else starting with 0 is routed over H.323
;^0\(.*\)$=h323/\1@long.distance.gateway
;   route short 3digit numbers to SIP using a DNS scheme 123 -> 3.2.1.domain
;^\(.\)\(.\)\(.\)$=sip/sip:\0@\3.\2.\1.domain
;   route only calls from SIP starting with 123 to a H.323 gateway
;${id}^sip/=if ^123.*$=h323/\0@provider.gw
;   is there anything else left? they go on E1 but only 15 channels can be used
;   we also make sure the number is at least 4 characters long
;   and we set a national caller dialplan
;.....*=zap/1-15 ; callerplan = national
;   leftovers... should not happen but let's handle them. we may not route the
;   call at all and let the caller receive a "no route" error
;.*=wave/play/sounds/invalid_number.gsm
;
; The following are for testing purposes
^99991001$=tone/dial
^99991002$=tone/busy
^99991003$=tone/ring
^99991004$=tone/specdial
^99991005$=tone/congestion
^99991006$=tone/outoforder
^99991007$=tone/milliwatt
^99991008$=tone/info

; Example of handling call authorization by caller authentication or ip address
; If the user is not authenticated call the subsection check_addr_auth
;${username}^$=call check_addr_auth
; Optionally, force caller id to authenticated username (if any)
;${username}.=;caller=${username}
;
;[check_addr_auth]
; Here we check for trusted gateways or networks by the "address" parameter
;  that for VoIP protocols is in the format: ip.ad.dr.ess:port
;
; allow trusted gateway 10.0.1.2
;${address}^10\.0\.1\.2:=return
; also trust callers from network 192.168.0.*
;${address}^192\.168\.0\.=return
; all others should be challenged (SIP,IAX) or rejected (other protocols)
;.*=-;error=noauth
|}


=== Parameters ===

This is a list of usual parameters found in the [[callroute|call.route]] message. Arbitrary parameters can be added by any module.
* ${address} - "address" (e.g. analog/pstn/1 or IP address) from which current incoming call was received.
* ${callsource} - e.g. fxs/fxo for analog lines
* ${formats} - list of codecs that the incoming call will accept
* ${id} - channel ID (e.g. 'sip/1')
* ${peerid} - peer's channel ID, usually empty during initial routing
* ${ip_host} - ip address of the transport connection or message
* ${ip_port} - port of the transport connection or message
* ${overlapped} - set by overlapped.php to indicated overlapped dialling is happening.
* ${rtp_forward} - do [[RTPForwarding|rtp forwarding]], can be no, possible, or yes
* ${type} - call type (e.g. record)
* ${username} - username for incoming call, only if authenticated
* ${caller} - caller ID (e.g. 'user' for a call from 'sip:user@example.com')
* ${callername} - caller ID name field
* ${called} - called party