URL Filter with Internal Data Base

Summary

The URL filter is a functionality that has the ability to restrict URLs accessible over HTTP communications. The "URL Filter with Internal Data Base" functionality allows you to register all or a part of a URL as a keyword and restrict access to URLs containing that keyword. In addition, by designating the source IP address when setting the filter, you can also limit connections from specific hosts or networks.

The primary specifications are as follows.

  • HTTP v. 1.0 and v. 1.1 are supported.
  • In the interface where you want to use a URL filter, when the destination port no. in the direction applicable to the URL filter and the port no. set with the "url filter port" command are a match, HTTP packets will be processed on the normal path.
  • You can output the filtering application results to a log or obtain statistical information via SNMP or output by command.
  • This functionality does not operate when the destination IP address of packets is the router address.

Notes

  • To restrict HTTPS communications, use a HTTPS URL Filter via Proxy. You cannot limit direct connections.
  • Internationalized domain names (IDN) are not supported.
  • For details on URLs, refer to RFC3986.
  • For details on HTTP, refer to RFC2616.

Compatible Models & Firmware Revisions

Model Firmware
RTX5000 Rev.14.00.15 or later
FWX120 Rev.11.03.16 or later
RTX810 Rev.11.01.15 or later

Details

Apply URL filters to the interface

  • You can set individual URL Filter criteria or IN/OUT in all directions.
  • When restricting web access from clients inside a network, set in the uplink direction to WAN (LAN side interface is IN, and WAN side interface is OUT).

Form URLs for evaluation

  • If unreserved characters as per RFC3986 are percent-encoded in a URL included in an HTTP packet, they are filtered and evaluated after decoding.
    Unreserved characters are: single-byte alphanumeric, "-" (hyphen), "_" (underscore), "."(period), and "~" (tilde).
  • When dot segments ("." or "..") are included in a path included in an HTTP packet, filtering and evaluation are done after the path has been interpreted and formed.

URL filter scanning

  • When there is a match at the reject filter, that packet is discarded, and the block screen is displayed on the client, and TCP RST is sent to the HTTP server. With the "url filter reject" command, you can set whether to return TCP RST to the client or send the block screen.
  • The process of URL filter evaluation follows URL filter registration number on "url INTERFACE filter" command.
    For example, when there are settings like,
    url filter 1 pass www.yamaha.co.jp
    url filter 2 reject yamaha
    url filter 3 pass *
    url lan2 filter out 1 2 3
    
    It allows to access http://www.yamaha.co.jp/, but not allow to access http://www.aaaa.com/yamaha/.
    If filter registration to the "url INTERFACE filter" command is reversed like,
    url lan2 filter out 2 1 3
    
    , access to both http://www.yamaha.co.jp/ and to http://www.aaaa.com/yamaha/ is rejected.
  • When you set a URL host name as a keyword in "url filter command", you cannot restrict access by HTTP to the IP address that resolves from the host name.
    Supposing the IP address of host www.yamaha.co.jp is 172.16.0.1.
    url filter 1 reject www.yamaha.co.jp
    url filter 2 pass *
    url lan2 filter out 1 2
    
    In the above settings, access to http://www.yamaha.co.jp/ is rejected but access to http://172.16.0.1/ is not.
  • When the keyword in a "url filter" command includes a URL with a percent encoded portion, the percent encoded portion will be handled as a character string as is, and not decoded.
    Set the keywords for the "url filter" command with decoded characters for percent encoded portions.
  • Access to a URL to which no set of keywords is applicable follows the setting that designates * (asterisk) as a keyword in the "url filter" command.
    If there is no setting to designate * (asterisk) as a keyword in the "url filter" command, unconditional rejection will occur.

MIB variables

You can acquire URL filter applied statistical information with the following MIB variables.

  • yamahaRTInterfaces
    Interface Information
    • yrIfUrlfilterTable
      Statistical information on URL Filters applied to LAN/PP/Tunnel Interfaces
      • yrIfUrlfilterEntry
        • yrIfUrlfilterIfIndex
          Interface Number
        • yrIfUrlfilterDir
          Filter Direction
        • yrIfUrlfilterId
          Filter ID Number
        • yrIfUrlfilterStatId
          Filter Matched Connection ID Number
        • yrIfUrlfilterSrcIP
          Filter Matched Connection Starting Point IP Address
        • yrIfUrlfilterCount
          Filter Match Instances

Command

URL Filter Configuration

[Syntax]
url filter ID KIND KEYWORD [SRC_ADDR[/MASK]]
no url filter ID
[Setting Value]
  • ID ... Filter number (1..65535)
  • KIND
    • pass, pass-nolog ..... Pass if matched (not record in the log)
    • pass-log ............. Pass if matched (record in the log)
    • reject, reject-log ... Discard if matched (record in the log)
    • reject-nolog ......... Discard if matched (not record in the log)
  • KEYWORD
    • Arbitrary string ... All or part of the URL to be filtered (up to 255 characters)
    • * ..... Apply to all URLs
  • SRC_ADDR ... Source IP address of the IP packet
    • Arbitrary IPv4 address ... A single IPv4 address
    • Rrange designation ... A range specified by two IP addresses separated by a hyphen or one IP address preceded or followed by a hyphen
    • * ..... Apply to all IP addresses
    • Omitted ..... Same as * when omitted.
  • MASK ... Netmask length (can be specified only when SRC_ADDR is a network address)
[Description]

Sets a URL filter. The filters set by this command are used by the "url INTERFACE filter" command.
If the specified keyword contains uppercase characters, they are converted to lowercase characters before the data is saved.

(For FWX120) To filter HTTPS connections through the proxy, note that you specify the URL (string) to be verified using keyword in the form of https://host-name:port-number.

[Applicable Models]
RTX5000 RTX810 FWX120

Apply a URL filter to an interface

[Syntax]
url INTERFACE filter DIR LIST
url pp filter DIR LIST
url tunnel filter DIR LIST
no url INTERFACE filter
no url pp filter
no url tunnel filter
[Setting Value]
  • INTERFACE ... LAN or WAN interface name
  • DIR
    • in ... Filter the HTTP input
    • out ... Filter the HTTP output
  • LIST ... Series of URL filter numbers delimited by spaces (up to 512 items for RTX5000, up to 128 items for all other models)
[Description]

Limits the HTTP packets that pass the interface by combining packet filters specified by the "url filter" command to reject specified URLs.

The number of settable filters is up to 512 on the RTX5000 models, and up to 128 on all other models. The command line character string length is up to 4095 characters.

Packets that do not meet any of the specified filters are discarded.

[Note]

RTX5000 does not support WAN interface for INTERFACE parameter.

[Applicable Models]
RTX5000 RTX810 FWX120

Set the HTTP Port Numbers to Apply the URL Filter To

[Syntax]
url filter port LIST
no url filter port
[Setting Value]
  • LIST ... Series of port numbers delimited by spaces (up to 4)
[Initial Value]
  • 80
[Description]

Sets the HTTP port numbers to apply the URL filter to.

[Applicable Models]
RTX5000 RTX810 FWX120

Show the URL Filter Information

[Syntax]
show url filter
show url filter INTERFACE
show url filter pp [PERR_NUM]
show url filter tunnel [TUNNEL_NUM]
[Setting Value]
  • INTERFACE ... LAN or WAN interface name
  • PEER_NUM ... Peer number
  • TUNNEL_NUM ... Tunnel interface number
[Description]

Shows statistical information about which of the specified interfaces matched with the filter and how many times.
If no interface is specified, the information for all interfaces is shown.

The following items are shown.

  • Filter number
  • Source IP address
  • The number of times that the HTTP connection matched with the filter
[Note]

If an asterisk is entered for both the keyword and the IP address in the "url filter" command, the number of times that the HTTP connection matched with that filter is not displayed.
RTX5000 does not support WAN interface for INTERFACE parameter.

[Applicable Models]
RTX5000 RTX810 FWX120

Clear the Statistical Information for the URL Filter

[Syntax]
clear url filter
clear url filter [INTERFACE]
clear url filter pp [PERR_NUM]
clear url filter tunnel [TUNNEL_NUM]
[Setting Value]
  • INTERFACE ... LAN or WAN interface name
  • PEER_NUM ... Peer number
  • TUNNEL_NUM ... Tunnel interface number
[Description]

Clears the statistical information for the URL filter. If no interface is specified, the information for all interfaces is cleared.

[Note]

RTX5000 does not support WAN interface for INTERFACE parameter.

[Applicable Models]
RTX5000 RTX810 FWX120

Set the HTTP Response to the Source of a Packet Discarded by the URL Filter

[Syntax]
url filter reject redirect
url filter reject redirect URL
url filter reject off
no url filter reject [ACTION]
[Setting Value]
  • redirect ... Return the HTTP redirect HTTP response and transfer it to the blocked item display
  • off ... Do not return an HTTP response. Use TCP RST to close the TCP session
  • url ... The URL to redirect to (up to 255 characters starting with "http://" or "https://")
  • action ...
    • redirect
    • off
[Initial Value]
  • redirect ... redirect (for all models except RTX5000)
  • off .... off (for RTX5000)
[Description]

Sets the HTTP response to the source of a packet discarded by the URL filter.
In the blocked item display, the filtered keyword and the reason that access was denied appear.

If a "url" was specified, when the URL is actually redirected, a question mark appears after the specified "url", and a query of the following type is appended.

  • The URL whose access was denied
  • The keyword setting of the applicable filter

You must set the "url" to a string that starts with "http://" or "https://".

[Note]

On models that support the HTTP server function, to set redirect and show the blocked item display on a Web browser, you must set "httpd service" on.

[Applicable Models]
RTX5000 RTX810 FWX120

Set Whether to Log Filter Matches

[Syntax]
url filter log SWITCH
no url filter log
[Setting Value]
  • SWITCH
    • on .... Log filter matches
    • off ... Do not log filter matches
[Initial Value]
  • on
[Description]

Sets whether to log filter matches.

[Note]

Even if you select on, logging does not take place for packets that match filters whose kind parameter has been set to pass, pass-nolog, or reject-nolog by the "url filter" command.

[Applicable Models]
RTX5000 RTX810 FWX120

Set Whether to Use the URL Filter

[Syntax]
url filter use SWITCH
no url filter use
[Setting Value]
  • SWITCH
    • on ... Use the URL filter.
    • off ... Do not use the URL filter.
[Initial Value]
  • on
[Description]

Sets whether to use the URL filter.

[Applicable Models]
RTX5000 RTX810 FWX120

Settings Examples

PP (1) interface settings example
url filter 1 reject * 192.168.0.100			... (1)
url filter 2 reject info *				... (2)
url filter 3 reject yamaha 192.168.0.2-192.168.0.10     ... (3)
url filter 4 reject http://aaa.bbb.ccc/ *	        ... (4)
url filter 1000 pass * *				... (5)
pp select 1
 url pp filter out 1 2 3 4 1000				... (6) 

(1)... Prohibit all access to URLs from hosts at 192.168.0.100.  
(2)... Prohibit access to URLs include info from all hosts. 
(3)... Prohibit access to URLs include yamaha from all hosts in the range of 192.168.0.2-192.168.0.10. 
(4)... Prohibit access to http://aaa.bbb.ccc/o from all hosts. 
(5)... Permit access to all URLs from all hosts. 
(6)... Apply filters (1)~(5) to HTTP connections on PP[1] in outbound direction. 
"show url filter" command display examples
# show url filter pp 1
pp 1 [OUT]:
filter id      source IPaddress     count
----------------------------------------
    1          192.168.0.10           29
    2          192.168.0.27           10
    3          192.168.0.74          917
    4          192.168.0.18           83

# 

When interface is not designated 
# show url filter
LAN 2 [OUT]:
filter id      source IPaddress     count
----------------------------------------
   19          192.168.100.231     83716
   22          192.168.100.18        378
   43          192.168.100.172      1058
   88          192.168.100.34       1892
  324          192.168.100.35       3871


pp 1 [OUT]:
filter id      source IPaddress     count
----------------------------------------
    1          192.168.0.10           29
    2          192.168.0.27           10
    3          192.168.0.74          917
    4          192.168.0.18           83

pp 2 [IN]:
filter id      source IPaddress     count
----------------------------------------
 2001          10.129.83.98         1238
 2002          172.16.38.137           9
 2003          10.211.49.176         328 

SYSLOG Message list

The SYSLOG messages output by this function are listed below. Note the prefix "[URL FILTER]" is added to the beginning of the output message.

Level Output message Content
NOTICE* Passed at INTERFACE DIR(NUM) filter: IP_ADDRESS : URL Packets to access from IP_ADDRESS to URL that match filter with filter number NUM, which was applied to DIR direction of INTERFACE, were passed.
Rejected at INTERFACE DIR(NUM) filter: IP_ADDRESS : URL Packets to access from IP_ADDRESS to URL that match filter with filter number NUM, which was applied to DIR direction of INTERFACE, were discarded.

* Output at DEBUG level on RTX810 and FWX120

Return to Top