From BroWiki

In this chapter we detail the different analyzers that Bro provides. Some analyzers look at traffic in fairly generic terms, such as at the level of TCP or UDP connections. Others delve into the specifics of a particular application that is carried on top of TCP or UDP.

As we use the term here, analyzer primarily refers to Bro's event engine. We use the term script to refer to a set of event handlers (and related functions and variables) written in the Bro language; module to refer to a script that serves primarily to provide utility (helper) functions and variables, rather than event handlers; and handler to denote an event handler written in the Bro language. Furthermore, the standard script is the script that comes with the Bro distribution for handling the events generated by a particular analyzer.

Note: However, we also sometimes use "analyzer" to refer to the event handler that processes events generated by the event engine.

We characterize the analyzers in terms of what events they generate, but don't here go into the details of how they generate the events (i.e., the nitty gritty C++ implementations of the analyzers).

Contents 1 Activating an Analyzer 1.1 Loading Analyzers 1.2 Filtering 2 Module Facility 3 General Processing Events 4 Generic Connection Analysis 4.1 The connection record 4.2 Definitions of connections 4.3 Generic TCP connection events 4.4 The tcp analyzer 4.5 The udp analyzer 4.6 Connection summaries 4.7 Connection functions 5 Site-specific information 5.1 Site variables 5.2 Site-specific functions 6 The hot Analyzer 6.1 hot variables 6.2 hot functions 7 The scan Analyzer 7.1 scan variables 7.2 scan functions 7.3 scan event handlers 8 The port-name Analysis Script 9 The brolite Analysis Script 10 The alarm Analysis Script 11 The active Analysis Script 12 The demux Analysis Script 13 The dns Analysis Script 13.1 The dns_mapping record 13.2 dns variables 13.3 dns event handlers 14 The finger Analyzer 14.1 finger variables 14.2 finger event handlers 15 The frag Analysis Script 16 The hot-ids Analysis Script 17 The ftp Analyzer 17.1 The ftp_session_info record 17.2 ftp variables 17.3 ftp functions 17.4 ftp event handlers 17.5 ftp notices 18 The http Analyzer 18.1 http variables 18.2 http event handlers 19 The ident Analyzer 19.1 ident variables 19.2 ident event handlers 20 The irc Analyzer 20.1 irc records 20.2 irc variables 20.3 irc event handlers 21 The login Analyzer 21.1 login analyzer confusion 21.2 login variables 21.3 login functions 21.4 login event handlers 22 The pop3 Analyzer 22.1 The pop3_session_info record 22.2 pop3 variables 22.3 pop3 event handlers 23 The portmapper Analyzer 23.1 portmapper variables 23.2 portmapper functions 23.3 portmapper event handlers 24 The analy Analyzer 25 The signature Analysis Script 26 The SSL Analyzer 26.1 The x509 record 26.2 The ssl_connection_info record 26.3 SSL variables 26.4 SSL event handlers 27 The weird Analysis Script 27.1 Actions for "weird" events 27.2 weird variables 27.3 weird functions 27.4 Events handled by conn_weird 27.5 Events handled by conn_weird_addl 27.6 Events handled by flow_weird 27.7 Events handled by net_weird 27.8 Events generated by the standard scripts 27.9 Additional handlers for ``weird events 28 The icmp Analyzer 29 The stepping Analyzer 30 The ssh-stepping Analysis Script 31 The backdoor Analyzer 32 The interconn Analyzer

Activating an Analyzer

In general, Bro will only do the work associated with a particular analyzer if your policy script defines one or more event handlers associated with the analyzer. For example, Bro will instantiate an FTP analyzer only if your script defines an ftp_request or ftp_reply handler. If it doesn't, then when a new FTP connection begins, Bro will only instantiate a generic TCP analyzer for it. This is an important point, because some analyzers can require Bro to capture a large volume of traffic (See Filtering) and perform a lot of computation; therefore, you need to have a way to trade off between the type of analysis you do and the performance requirements it entails, so you can strike the best balance for your particular monitoring needs.

Deficiency: While Bro attempts to instantiate an analyzer if you define a handler for any of the events the analyzer generates, its method for doing so is incomplete: if you only define an analyzer's less mainstream handlers, Bro may fail to instantiate the analyzer.

Loading Analyzers

The simplest way to use an analyzer is to @load the standard script associated with the analyzer. (See load directive for a discussion of @load). However, there's nothing magic about these scripts; you can freely modify or write your own. The only caveat is that some scripts @load other scripts, so the original version may wind up being loaded even though you've also written your own version.

Deficiency: It would be useful to have a mechanism to fully override one script with another.

In this chapter we discuss each of the standard scripts as we discuss their associated analyzers.

Filtering

Most analyzers require Bro to capture a particular type of network traffic. These traffic flows can vary immensely in volume, so different analyzers can cost greatly differing amounts in terms of performance. Bro declares two redefinable tables in pcap.bro that have special interpretations with regard to filtering:

   global capture_filters: table[string] of string &redef;
   global restrict_filters: table[string] of string &redef;

The key strings serve as a user-definable identifier for the filter strings they are associated with. The entries of the capture_filters table define what traffic Bro should capture, while restrict_filters' entries limit what traffic Bro captures. Bro builds the following tcpdump filter from both tables:

('OR' of capture_filters' entries) and ('AND' of restrict_filters' entries)

Thus, repeated Refinements of capture_filters using the += initializer are combined using logical "OR"s, whereas for restrict_filters "AND"s are used. This follows from the tables' respective purposes---capture_filters permits any of its components, while restrict_filters rejects everything that does not comply with all of its components.

If you do not define capture_filters, then its value is set to tcp or udp; if you do not define restrict_filters, then no restriction is in effect.

Here is an example. If you specify:

   redef capture_filters = { ["HTTP"] = "port http" };
   redef restrict_filter = { ["mynet"] = "net 128.3" };

then the corresponding tcpdump filter will be:

   (port http) and (net 128.3)

which will capture only the TCP port 80 traffic that has either a source or destination address belonging to the 128.3 network (i.e., 128.3/16). A more complex example:

   redef capture_filters += { ["DNS"] = "udp port 53" };
   redef capture_filters += { ["FTP"] = "port ftp" };

   redef restrict_filters += { ["foonet"] = "net 128.3" };
   redef restrict_filters += { ["noflood"] = "not host syn-flood.magnet.com" };

yields this tcpdump filter:

   ((udp port 53) or (port ftp)) and ((net 128.3) and (not host syn-flood.magnet.com))

As you add analyzers, the final tcpdump filter can become quite complicated. You can load the predefined print-filter script to print out the resulting filter. This script handles the bro_init event and exits Bro after printing the filter. Its intended use is that you can add it to the Bro command line (bro my-own-script print-filter) when you want to see what filter the script my-own-script winds up using.

There are two particular uses for print-filter. The first is to debug filtering problems. Unfortunately, Bro sometimes uses sufficiently complicated expressions that they tickle bugs in tcpdump's optimizer. You can take the filter printed out for your script and try running it through tcpdump by hand, and then also try using tcpdump's -O option to see if turning off the optimizer fixes the problem. The second use is to provide a shadow backup to Bro: that is, a version of tcpdump running either on the same machine or a separate machine that uses the same network filter as Bro. While tcpdump can't perform any analysis of the traffic, the shadow guards against the possibility of Bro crashing, because if it does, you will still have a record of the subsequent network traffic which you can run through Bro for post-analysis.

Module Facility

The module facility implements namespaces. Everything is in some namespace or other. The default namespace is called "GLOBAL" and is searched by default when doing name resolution. The scoping operator is "::" as in C++. You can only access things in the current namespace, things in the GLOBAL namespace, or things that have been explicitly exported from a different namespace. Exported variables and functions still require fully-qualified names. The syntax is as follows:

module foo;  # Sets the current namespace to "foo"
 export {
   int i;
   int j;
 }
 int k;

module bar;
 int i;

 foo::i = 1;
 bar::i = 2;
 print i;    # bar::i (since we're currently in module bar)
 j = 3;      # ERROR: j is exported, but the fully qualified name
             #        foo::j is required
 foo::k = 4; # ERROR: k is not exported

The same goes for calling functions.

One restriction currently in place is that variables not in the "GLOBAL" namespace can't shadow those in GLOBAL, so you can't have:

   module GLOBAL;
   global i: int;

   module other_module;
   global i: int;

It is a little confusing that the "global" declaration really only means that the variable i is global to the current module, not that it is truly global and thus visible everywhere (that would require that it be in GLOBAL, or if using the full name is okay, that it be exported). Perhaps there will be a change to the syntax in the future to address this.

The "module" statement cuts across @load commands, so that if you say:

   module foo;
   @load other_script;

then other_script will be in module foo. Likewise if other_script changes to module bar, then the current module will be module bar even after other_script is done. However, this functionality may change in the future if it proves problematic.

The policy scripts in the Bro distribution have not yet been updated to use it, but there is a backward-compatibility feature so that existing scripts should work without modification. In particular, everything is put in GLOBAL by default.

General Processing Events

Bro provides the following events relating to its overall processing:

• bro_init () is generated when Bro first starts up. In particular, after Bro has initialized the network (or initialized to read from a save file) and executed any initializations and global statements, and just before Bro begins to read packets from the network input source(s).

• net_done (t: time) generated when Bro has finished reading from the network, due to either having exhausted reading the save file(s), or having received a terminating signal (See General Processing Events). t gives the time at which network processing finished. This event is generated before bro_done. Note: If Bro terminates due to an invocation of exit, then this event is not generated.

Deficiency: This event is generated on a terminating signal even if Bro is not reading network traffic.

• bro_done () generated when Bro is about to terminate, either due to having exhausted reading the save file(s), receiving a terminating signal (See General Processing Events), or because Bro was run without the network input source and has finished executing any global statements.
  This event is generated after net_done. If you have cleanup that only needs to be done when processing network traffic, it likely is better done using net_done. Note: If Bro terminates due to an invocation of exit, then this event is not generated.

• bro_signal (signal: count) generated when Bro receives a signal. Currently, the signals Bro handles are SIGTERM, SIGINT, and SIGHUP.
  Receiving either of the first two terminates Bro, though if Bro is in the middle of processing a set of events, it first finishes with them before shutting down. The shutdown leads to invocations of net_done and bro_done, in that order. Upon receiving SIGHUP, Bro invokes flush_all (in addition to your handler, if any).

Deficiency: When receiving a signal during event processing, Bro fails to invoke bro_signal, clearly a bug.

• net_stats_update (t: time, ns: net_stats) This event includes two arguments, t, the time at which the event was generated, and ns, a net_stats record, as defined in the example below. Regarding this second parameter, the pkts_recvd field gives the total number of packets accepted by the packet filter so far during this execution of Bro; pkts_dropped gives the total number of packets reported dropped by the kernel; and interface_drops gives the total number of packets reported by the kernel as having been dropped by the network interface.
  Note: An important consideration is that, as shown by experience, the kernel's reporting of these statistics is not always accurate. In particular, the $pkts_dropped statistic is sometimes missing actual packet drops, and some operating systems do not support the interface_drops statistic at all. See the ack_above_hole event for an alternate way to detect if packets are being dropped.

type net_stats: record {
   # All counts are cumulative.
   pkts_recvd: count;       # Number of packets received so far.
   pkts_dropped: count;     # Number of packets *reported* dropped.
   interface_drops: count;  # Number of drops reported by interface(s).
};

Generic Connection Analysis

The conn analyzer performs generic connection analysis: connection start time, duration, sizes, hosts, and the like. You don't in general load analyzer directly, but instead do so implicitly by loading the tcp, udp, or icmp analyzers. Consequently, analyzer doesn't configure capture_filters by itself, but instead uses whatever is set up by these more specific analyzers.

conn analyzes a number of events related to connections beginning or ending. We first describe the connection record data type that keeps track of the state associated with each connection (See connection record), and then we detail the events in Generic TCP connection events. The main output of its analysis are one-line connection summaries, and in Connection functions we give an overview of the different callable functions provided by conn.

conn also loads three other Bro modules: the hot and scan analyzers, and the port_name utility module.

The connection record

type conn_id: record {
   orig_h: addr;  # Address of originating host.
   orig_p: port;  # Port used by originator.
   resp_h: addr;  # Address of responding host.
   resp_p: port;  # Port used by responder.
};

type endpoint: record {
   size: count;  # Bytes sent by this endpoint so far.
   state: count; # The endpoint's current state.
};

type connection: record {
   id: conn_id;        # Originator/responder addresses/ports.
   orig: endpoint;     # Endpoint info for originator.
   resp: endpoint;     # Endpoint info for responder.
   start_time: time;   # When the connection began.
   duration: interval; # How long it was active (or has been so far).
   service: string;    # The service we associate with it (e.g., "http").
   addl: string;       # Additional information associated with it.
   hot: count;         # How many times we've marked it as sensitive.
};

A connection record record holds the state associated with a connection, as shown in the example above. Its first field, id, is defined in terms of the conn_id record, which has the following fields:

• orig_h: The IP address of the host that originated (initiated) the connection. In "client/server" terminology, this is the "client."

• orig_p: The TCP or UDP port used by the connection originator (client). For ICMP "connections," it is set to 0 icmp Analyzer.

• resp_h: The IP address of the host that responded (received) the connection. In "client/server" terminology, this is the "server."

• resp_p: The TCP or UDP port used by the connection responder (server). For ICMP "connections," it is set to 0 icmp Analyzer.

The orig and resp fields of a connection record both hold endpoint record values, which consist of the following fields:

• size: How many bytes the given endpoint has transmitted so far. Note that for some types of filtering, the size will be zero until the connection terminates, because the nature of the filtering is to discard the connection's intermediary packets and only capture its start/stop packets.

• state: The current state the endpoint is in with respect to the connection. The table below defines the different possible states for TCP and UDP connections.

Deficiency: The states are currently defined as count, but should instead be an enumerated type; but Bro does not yet support enumerated types.

Note: UDP "connections" do not have a well-defined structure, so the states for them are quite simplistic. See Definitions of connections for further discussion.

The remaining fields in a connection record are:

• start_time: The time at which the first packet associated with this connection was seen.

• duration : How long the connection lasted, or, if it is still active, how long since it began.

• service: The name of the service associated with the connection. For example, if $id$resp_p is tcp/80, then the service will be "http". Usually, this mapping is provided by the global variable, perhaps via the endpoint_id function; but the service does not always directly correspond to $id$resp_p, which is why it's a separate field. In particular, an FTP data connection can have a service of "ftp-data" even though its $id$resp_p is something other than tcp/20 (which is not consistently used by FTP servers). If the name of the service has not yet been determined, then this field is set to an empty string.

• addl: Additional information associated with the connection. For example, for a login connection, this is the username associated with the login. If no additional information is yet associated with this connection, then this field is set to an empty string.

Deficiency: A significant deficiency associated with the addl field is that it is simply a string without any further structure. In practice, this has proven too restrictive. For example, we may well want to associate an unambiguous username with a login session, and also keep track of the names associated with failed login attempts. (See the login analyzer for an example of how this is implemented presently.) What's needed is a notion of union types which can then take on a variety of values in a type-safe manner.

• hot: How many times this connection has been marked as potentially sensitive or reflecting a break-in. The default value of 0 means that so far the connection has not been regarded as ``hot.

Note: Bro does not presently make fine-grained use of this field; the standard scripts alarm on connections with a non-zero hot field, and do not in general alarm on those that do not, though there are exceptions. In particular, the hot field is not rigorously maintained as an indicator of trouble; it instead is used loosely as an indicator of particular types of trouble (access to sensitive hosts or usernames).

Definitions of connections

Connections for TCP are well-defined, because establishing and terminating a connection plays a central part of the TCP protocol. Beyond those, Bro enforces a hard connection timeout after the period of time specified through the tcp_inactivity_timeout variable, defined in bro.init.

For UDP, a connection begins when host A sends a packet to host B for the first time, B never having sent anything to A. This transmission is termed a request, even if in fact the application protocol being used is not based on requests and replies. If B sends a packet back, then that packet is termed a reply. Each packet A or B sends is another request or reply. UDP connection timeouts are specified through the udp_inactivity_timeout variable, defined in bro.init.

For ICMP, Bro likewise creates a connection the first time it sees an ICMP packet from A to B, even if B previously sent a packet to A, because that earlier packet would have been for a different transport connection than the ICMP itself---the ICMP will likely refer to that connection, but it itself is not part of the connection. For simplicity, this holds even for ICMP ECHOs and ECHO_REPLYs; if you want to pair them up, you need to do so explicitly in the policy script. ICMP connection timeouts are specified through the icmp_inactivity_timeout variable, defined in bro.init.

Generic TCP connection events

There are a number of generic events associated with TCP connections, all of which have a single connection record as their argument:

• new_connection: Generated whenever state for a new (TCP) connection is instantiated.

Note: Handling this event is potentially expensive. For example, during a SYN flooding attack, every spoofed SYN packet will lead to a new new_connection event.

• connection_established: Generated when a connection has become established, i.e., both participating endpoints have agreed to open the connection.

• connection_attempt: Generated when the originator (client) has unsuccessfully attempted to establish a connection. "Unsuccessful" is defined as at least ATTEMPT_INTERVAL seconds having elapsed since the client first sent a connection establishment packet to the responder (server), where ATTEMPT_INTERVAL is an internal Bro variable which is presently set to 300 seconds.

Deficiency: This variable should be user-settable. If you want to immediately detect that a client is attempting to connect to a server, regardless of whether it may soon succeed, then you want to handle the new_connection event instead.

Note: Handling this event is potentially expensive. For example, during a SYN flooding attack, every spoofed SYN packet will lead to a new connection_attempt event, albeit delayed by ATTEMPT_INTERVAL.

• partial_connection: Generated when both connection endpoints enter the TCP_PARTIAL state This means that we have seen traffic generated by each endpoint, but the activity did not begin with the usual connection establishment.

Deficiency: For completeness, Bro's event engine should generate another form of partial_connection event when a single endpoint becomes active (see new_connection below). This hasn't been implemented because our experience is network traffic often contains a great deal of "crud", which would lead to a large number of these really-partial events. However, by not providing the event handler, we miss an opportunity to detect certain forms of stealth scans until they begin to elicit some form of reply.

TCP and UDP connection states, as stored in an endpoint record

State	Meaning
TCP_INACTIVE	The endpoint has not sent any traffic.
TCP_SYN_SENT	It has sent a SYN to initiated a connection.
TCP_SYN_ACK_SENT	It has sent a SYN ACK to respond to a connection request.
TCP_PARTIAL	The endpoint has been active, but we did not see the beginning of the connection.
TCP_ESTABLISHED	The two endpoints have established a connection.
TCP_CLOSED	The endpoint has sent a FIN in order to close its end of the connection.
TCP_RESET	The endpoint has sent a RST to abruptly terminate the connection.
UDP_INACTIVE	The endpoint has not sent any traffic.
UDP_ACTIVE	The endpoint has sent some traffic.

• connection_finished: Generated when a connection has gracefully closed.

Note: This event is triggered after the second FIN has been seen, not the potential final ACK following it. In case you see new_connection events for the final ACK, make sure that tcp_close_delay is large enough (for example loading the heavy-analysis.bro policy, which bumps up several timeout values).

• connection_rejected: Generated when a server rejects a connection attempt by a client.

Note: This event is only generated as the client attempts to establish a connection. If the server instead accepts the connection and then later aborts it, a connection_reset event is generated (see below). This can happen, for example, due to use of TCP Wrappers.

Note: Per the discussion above, a client attempting to connect to a server will result in one of connection_attempt, connection_established, or connection_rejected; they are mutually exclusive.

• connection_half_finished : Generated when Bro sees one endpoint of a connection attempt to gracefully close the connection, but the other endpoint is in the TCP_INACTIVE state. This can happen due to split routing, in which Bro only sees one side of a connection.

• connection_reset: Generated when one endpoint of an established connection terminates the connection abruptly by sending a TCP RST packet.

• connection_partial_close : Generated when a previously inactive endpoint attempts to close a connection via a normal FIN handshake or an abort RST sequence. When it sends one of these packets, Bro waits PARTIAL_CLOSE_INTERVAL (an internal Bro variable set to 10 seconds) prior to generating the event, to give the other endpoint a chance to close the connection normally.

• connection_pending: Generated for each still-open connection when Bro terminates.

The tcp analyzer

The general tcp analyzer lets you specify that you're interested in generic connection analysis for TCP. It simply @load's conn and adds the following to :

   tcp[13] & 0x7 != 0

which instructs Bro to capture all TCP SYN, FIN and RST packets; that is, the control packets that delineate the beginning (SYN) and end (FIN) or abnormal termination (RST) of a connection.

The udp analyzer

The general udp analyzer lets you specify that you're interested in generic connection analysis for UDP. It @load's both hot and conn, and defines two event handlers:

• udp_request (u: connection): Invoked whenever a UDP packet is seen on the forward (request) direction of a UDP connection. See Definitions of connections for a discussion of how Bro defines UDP connections.
  The analyzer invokes check_hot with a mode of CONN_ATTEMPTED and then record_connections to generate a connection summary (necessary because Bro does not time out UDP connections, and hence cannot generate a connection-attempt-failed event).

• udp_reply (u: connection): Invoked whenever a UDP packet is seen on the reverse (reply) direction of a UDP connection. See Definitions of connections for a discussion of how Bro defines UDP connections.
  The analyzer invokes check_hot with a mode of CONN_ESTABLISHED and then again with a mode of CONN_FINISHED to cover the general case that the reply reflects that the connection was both established and is now complete. Finally, it invokes to generate a connection summary.

Note: The standard script does not update capture_filters to capture UDP traffic. Unlike for TCP, where there is a natural generic filter that captures only a subset of the traffic, the only natural UDP filter would be simply to capture all UDP traffic, and that can often be a huge load.

Connection summaries

The main output of conn is a one-line ASCII summary of each connection. By tradition, these summaries are written to a file with the name conn.tag.log, where tag uniquely identifies the Bro session generating the logs.

The summaries are produced by the record_connection function, and have the following format (all reported on a single line):

 <start> <duration> <local IP> <remote IP> <service> <local port> \
 <remote port> <protocol> <org bytes sent>, <res bytes sent> <state> \
 <flags> <tag>

• start: corresponds to the connection's start time, as defined by start_time. * duration: gives the connection's duration, as defined by duration. * local IP, remote IP: correspond to the local and remote addresses that participated in the connection, respectively. The notion of which addresses are local is controlled by the global variable local_nets, which has a default value of empty. If local_nets has not been redefined, then local IP is the connection responder and remote IP is the connection originator. * service: is the connection's service, as defined by service. * local port, remote port: are the ports used by the connection. * org bytes sent res bytes sent: give the number of bytes sent by the originator and responder, respectively. These correspond to the size fields of the corresponding endpoint records. * state: reflects the state of the connection at the time the summary was written (which is usually either when the connection terminated, or when Bro terminated). The different states are summarized in the table below.

Summaries of connection states, as reported in conn.log files

Name	Meaning
S0	Connection attempt seen, no reply.
S1	Connection established, not terminated.
SF	Normal establishment and termination. Note that this is the same symbol as for state S1. You can tell the two apart because for S1 there will not be any byte counts in the summary, while for SF there will be.
REJ	Connection attempt rejected.
S2	Connection established and close attempt by originator seen (but no reply from responder).
S3	Connection established and close attempt by responder seen (but no reply from originator).
RSTO	Connection established, originator aborted (sent a RST).
RSTR	Established, responder aborted.
RSTOS0	Originator sent a SYN followed by a RST, we never saw a SYN-ACK from the responder.
RSTRH	Responder sent a SYN ACK followed by a RST, we never saw a SYN from the (purported) originator.
SH	Originator sent a SYN followed by a FIN, we never saw a SYN ACK from the responder (hence the connection was "half" open).
SHR	Responder sent a SYN ACK followed by a FIN, we never saw a SYN from the originator.
OTH	No SYN seen, just midstream traffic (a "partial connection" that was not later closed).

The ASCII Name given in the Table is what appears in the conn.tag.log log file; it is returned by the conn_state function. The Symbol is used when generating human-readable versions of the file---see hot-report script.

For UDP connections, the analyzer reports connections for which both endpoints have been active as SF; those for which just the originator was active as S0; those for which just the responder was active as SHR; and those for which neither was active as OTH (this latter shouldn't happen!).

• flags: reports a set of additional binary state associated with the connection: ** L indicates that the connection was initiated locally, i.e., the host corresponding to A_l initiated the connection. If L is missing, then the host corresponding to A_r initiated the connection. ** U indicates the connection involved one of the networks listed in the neighbor_nets variable. The use of U for this indication (rather than N, say) is historical, as for the most part is the whole notion of "neighbor network." Note that connection can have both L and U set (see next item). ** X is used to indicate that neither the L or U flags is associated with this connection. * tag: Reference tag to log lines containing additional information associated with the connection in other log files, (e.g.: http.log).

Putting all of this together, here is an example of a conn.log connection summary:

931803523.006848 54.3776 http 7320 38891 206.132.179.35 128.32.162.134 RSTO X %103

The connection began at timestamp 931803523.006848 (18:18:43 hours GMT on July 12, 1999; see the cf utility for how to determine this) and lasted 54.3776 seconds. The service was HTTP (presumably; this conclusion is based just on the responder's use of port 80/tcp). The originator sent 7,320 bytes, and the responder sent 38,891 bytes. Because the L flag is absent, the connection was initiated by host 128.32.162.134, and the responding host was 206.132.179.35. When the summary was written, the connection was in the RSTO state, i.e., after establishing the connection and transferring data, the originator had terminated it with a RST (this is unfortunately common for Web clients). The connection had neither the L or U flags associated with it, and there was additional information, summarized by the string %103 (see the http analyzer for an explanation of this information).

Connection functions

We finish our discussion of generic connection analysis with a brief summary of the different Bro functions provided by the conn analyzer:

• conn_size (e: endpoint, is_tcp: bool): string returns a string giving either the number of bytes the endpoint sent during the given connection, or "?" if from the connection state this can't be determined. The is_tcp parameter is needed so that the function can inspect the endpoint's state to determine whether the connection was closed.

• conn_state (c: connection, is_tcp: bool): string returns the name associated with the connection's state, as given in the above table.

• determine_service (c: connection): bool sets the service field of the given connection, using port_names. If you are using the ftp analyzer, then it knows about FTP data connections and maps them to port_names[20/tcp], i.e., "ftp-data".

• full_id_string (c: connection): string returns a string identifying the connection in one of the two following forms. If the connection is in state S0, S1, or REJ, then no data has been transferred, and the format is:

 A_o <state> A_r/<service> <addl>

where A_o is the IP address of the originator ($id$orig_h), state is as given in the Symbol column of the above table. A_r is the IP address of the responder ($id$resp_h), service gives the application service ($service) as set by determine_service, and addl is the contents of the $addl field (which may be an empty string).

Note that the ephemeral port used by the originator is not reported. If you want to display it, use id_string.

So, for example:

 128.3.6.55 > 131.243.88.10/telnet "luser"

identifies a connection originated by 128.3.6.55 to 131.243.88.10's Telnet server, for which the additional associated information is "luser", the username successfully used during the authentication dialog as determined by the analyzer. From the table above we see that the connection must be in state S1, as that's the only state of S0, S1, or REJ that has a > symbol. (We can tell it's not in state SF because the format used for that state differs---see below.)

For connections in other states, Bro has size and duration information available, and the format returned by full_id_string is:

 A_o S_o <state> A_r/<service> S_r D_s <addl>

where A_o, A_r, state, service, and addl are as before, S_o and S_r give the number of bytes transmitted so far by the originator to the responder and vice versa, and D gives the duration of the connection in seconds (reported with one decimal place) so far.

An example of this second format is:

   128.3.6.55 63b > 131.243.88.10/telnet 391b 39.1s "luser"

which reflects the same connection as before, but now 128.3.6.55 has transmitted 63 bytes to 131.243.88.10, which has transmitted 391 bytes in response, and the connection has been active for 39.1 seconds. The ">" indicates that the connection is in state SF.

• id_string (id: conn_id): string returns a string identifying the connection by its address/port quadruple. Regardless of the connection's state, the format is:

 A_o / P_o  > A_r / P_r

where A_o and A_r are the originator and responder addresses, respectively, and P_o and P_r are representations of the originator and responder ports as returned by the port-name module, i.e., either or a string like "http" for a well-known port such as 80/tcp.

An example:

 128.3.6.55/2244 > 131.243.88.10/telnet

Note, id_string is implemented using a pair of calls to endpoint_id.

Deficiency: It would be convenient to have a form of id_string that can incorporate a notion of directionality, for example 128.3.6.55/2244 < 131.243.88.10/telnet to indicate the same connection as before, but referring specifically to the flow from responder to originator in that connection (indicated by using "<" instead of ">").

• log_hot_conn (c: connection) logs a real-time SensitiveConnection alarm of the form:

 hot:  < connection-id >

where connection-id is the format returned by full_id_string. log_hot_conn keeps track of which connections it has logged and will not log the same connection more than once.

• record_connection (c: connection, disposition: string) Generates a connection summary to the @file{conn} file in the format described in Connection summaries. If the connection's hot field is positive, then also logs the connection using log_hot_conn. The disposition is a text description of the connection's state, such as "attempt" or "half_finished"; it is not presently used.

• service_name (c: connection): string returns a string describing the service associated with the connection, computed as follows. If the responder port ($id$resp_p), p, is well-known, that is, in the port_names table, then p's entry in the table is returned (such as "http" for TCP port 80). Otherwise, for TCP connections, if the responder port is less than 1024, then priv-p is returned, otherwise other-p. For UDP connections, the corresponding service names are upriv-p and uother-p.

• terminate_connection (c: connection) Attempts to terminate the given connection using the rst utility in the current directory. It does not check to see whether the utility is actually present, so an unaesthetic shell error will appear if the utility is not available. rst terminates connections by forging RST packets. It is not presently distributed with Bro, due to its potential for disruptive use.

If Bro is reading a trace file rather than live network traffic, then terminate_connection logs the rst invocation but does not actually invoke the utility. In either case, it finishes by logging that the connection is being terminated.

Site-specific information

The site analyzer is not actually an analyzer but simply a set of global variables (and Updateme: one function) used to define a site's basic topological information.

Site variables

The site module defines the following variables, all redefinable:

• local_nets set[net]: Defines which net's Bro should consider as reflecting a local address. Default: empty.

• local_16_nets set[net]: Defines which /16 prefixes Bro should consider as reflecting a local address. Default: empty.

Deficiency: Bro currently is inconsistent regarding when it consults local_nets versus local_16_nets, so you should ensure that this variable and the previous one are always consistent.

• local_24_nets set[net]: The same, but for /24 addresses. Default: empty.

• neighbor_nets set[net]: Defines which net's Bro should consider as reflecting a "neighbor." Neighbors networks can be treated specially in some policies, distinct from other non-local addresses. In particular, will not drop connectivity to an address belonging to a neighbor.
  The notion is somewhat historical, as is the use of ``U to mark neighbors in connection summaries (See Connection summaries). Default: empty.

• neighbor_16_nets set[addr]: Defines which /16 addresses Bro should consider as reflecting a neighbor; the only use of this variable in the standard scripts is that a scan originating from an address with one of these prefixes will not be dropped. Default: empty.

Deficiency: The name is poorly chosen and should be changed to better reflect this use.

Deficiency: In addition, this variable should be kept consistent with neighbor_nets, until the fine day when the processing is rectified to only use one variable.

• neighbor_24_nets set[net]: The same, but for /24 addresses. Default: empty.

Site-specific functions

Currently, the site module only defines one function:

• is_local_addr (a: addr): bool returns true if the given address belongs to one of the "local" networks, false otherwise.

Fixme: Currently, the test is made by masking the address to /16 and /24 and comparing it to local_16_nets and local_24_nets.

The hot Analyzer

The standard hot script defines policy relating to fairly generic notions of allowed and prohibited connections. It defines a number of variables that you will need to refine to customize your site's policies. It also provides two functions for checking connections against the policies, which can be used by other of the standard scripts.

hot variables

The standard hot script defines the following variables, all redefinable:

• same_local_net_is_spoof : bool If true, then a connection with a local originator address and a local responder address is considered by to have been spoofed.

Deficiency: The name is poorly chosen (and may be changed in the future) to something more accurate like both_local_nets_is_spoof.

In general, you want to use true for a Bro that is monitoring Internet access links (DMZs) and false for internal monitors. Default: F.

• allow_spoof_services : set[port]: Defines a set of services (responder ports) for which Bro should not generate notices if it sees apparent spoofed traffic. Default: 110/tcp (POP version 3; RFC-1939). This default was chosen because in our experience one common form of benign spoof is an off-site laptop attempting to read mail while still configured to use its on-site address.

• allow_pairs : set[addr, addr] Defines pairs of source and destination addresses for which the source is allowed to connect to the destination. The intent with this variable is that the source or destination address will be a sensitive host (such as defined with host_src or host_dsts), for which this particular access should be allowed. Default: empty.

• allow_16_net_pairs : set[addr, addr] Defines pairs of source and destination /16 networks for which the source is allowed to connect to the destination, similar to allow_pairs. Default: empty.

Note: The set is defined in terms of addr's and not net's. So, for example, rather than specifying 128.32., which is a net constant, you'd use 128.32.0.0 (an addr constant).

• hot_srcs : table[addr] of string Defines source addresses that should be considered "hot." A successfully established connection from such a source address generates an alarm, unless one of the access exception variables such as allow_pairs also matches the connection. The value of the table gives an explanatory message as to why the source is hot; for example, "known attacker site". Default: empty.

Note: This value is not currently used, though it aids in documenting the policy script.

Example: redefining hot_srcs using

redef hot_srcs: table[addr] of string = {
   [ph33r.the.eleet.com] = "script kideez",
};

would result in Bro noticing any traffic coming ph33r.the.eleet.com.

• hot_dsts : table[addr] of string Same as hot_srcs, except for destination addresses. Default: empty.

• hot_src_24nets : table[addr] of string Defines /24 source networks should be considered "hot," similar to hot_srcs. Default: empty.

Deficiency: Other network masks, particularly /16, should be provided.

Example: redefining hot_src_24nets using

redef hot_src_24nets: table[addr] of string = {
   [198.81.129.0] = "CIA incoming!",
};

would result in Bro noticing any traffic coming from the 198.81.129/24 network.

• hot_dst_24nets : table[addr] of string same as hot_src_24nets, except for destination networks. Default: empty.

• allow_services : set[port] Defines a set of services that are always allowed, regardless of whether the source or destination address is "hot." Default: ssh, http, gopher ident, smtp, 20/tcp (FTP data).

Note: The defaults are a bit unusual. They are intended for a quite open site with many services.

• allow_services_to : set[addr, port] Defines a set of services that are always allowed if the server is the given host, regardless of whether the source or destination address is "hot." Default: empty.

Example: redefining allow_services_to using

redef allow_services_to: set[addr, port] += {
   [ns.mydomain.com, [domain, 123/tcp]],
} &redef;

would result in Bro not noticing any TCP DNS or NTP traffic heading to ns.mydomain.com. You might add this if ns.mydomain.com is also in hot_dsts, because in general you want to consider any access (other than DNS or NTP) as sensitive.

• allow_services_pairs : set[addr, addr, port] Defines a set of services that are always allowed if the connection originator is the first address and the responder (server) the second address. Default: empty.

Example: redefining allow_services_pairs using

redef allow_services_pairs: set[addr, addr, port] += {
   [ns2.mydomain.com, ns.mydomain.com, [domain, 123/tcp]],
} &redef;

would result in Bro not noticing any TCP DNS or NTP traffic initiated from ns2.mydomain.com to ns.mydomain.com.

• flag_successful_service : table[port] of string The opposite of allow_services. Defines a set of services that should always be flagged as sensitive, even if neither the source nor the destination address is "hot." The string value in the table gives the reason for why the service is considered hot. Note: Bro currently does not use these explanatory messages. Default: 31337/tcp (a popular backdoor because in stylized lettering it spells ELEET) and 2766/tcp (the Solaris listen service, in our experience rarely used legitimately in wide-area traffic).

Note: Bro can flag these services erroneously when a server happens to run a different service on the same port. For example, if you're not running the FTP analyzer, then Bro won't know that FTP data connections using ephemeral ports in fact belong to legitimate FTP traffic, and will flag any that coincide with these services. A related problem arises when a user has configured their SSH access to tunnel FTP control channels through the FTP connection, but not the corresponding data connections (so they don't pay the expense of encrypting the data transfers), so again Bro can't recognize that the ephemeral ports used for the data connections does not reflect the presumed sensitive service.

Example: redefining flag_successful_service using

redef flag_successful_service: table[port] of string += {
       [1524/tcp] = "popular backdoor",
};

would result in Bro also noticing any successful connection to a server running on TCP port 1524.

• flag_successful_inbound_service : table[port] of string The same as flag_successful_service, except only applies to connections with a remote initiator and a local responder (determined by finding the responder address in local_nets). Default: 1524/tcp (ingreslock, a popular backdoor because an attacker can place an entry for the backdoor in /etc/inetd.conf using a service name rather than a raw port number, and hence more likely to appear legitimate to casual inspection). Note: There's no compelling reason why ingreslock is in this table rather than the more general flag_successful_service, though it does tend to result in a few more false hits than the others, presumably because it's a lower port number, and hence more likely on some systems to be chosen for an ephemeral port.

Note: Symmetry would call for flag_successful_outbound_service. This hasn't been implemented in Bro yet simply because the Bro development site has a threat model structured primarily around external threats.

• terminate_successful_inbound_service : table[port] of string The same as flag_successful_inbound_service, except invokes in an attempt to terminate the connection. Default: empty.

Note: As for flag_successful_inbound_service, it would be symmetric to have terminate_successful_outbound_service, and also to have a more general terminate_successful_service.

• flag_rejected_service table[port] of string Similar to flag_successful_service, except applies to connections that a server rejects. For example, you could detect a particular, failed Linux "mountd" attack by adding 10752/tcp to this table, since that happens to be the port used by the commonly available version of the exploit for its backdoor if the attack succeeds. Note: You would of course likely also want to put 10752/tcp in flag_successful_service; or put the entire flag_rejected_service table into flag_successful_service, as discussed in Inserting tables into tables. Default: none.

Deficiency: It might make sense to have flag_attempted_service, which doesn't require that a server actively reject the connection, but Bro doesn't currently have this.

hot functions

The hot module defines two functions for external use:

• check_spoof (c: connection): bool checks the originator and responder addresses of the given connection to determine if they are both local (and the connection is not explicitly allowed in allow_spoof_services). If so, and if same_local_net_is_spoof is true, then marks the connection as "hot".

The function also checks for a specific denial of service attack, the "Land" attack, in which the addresses are the same and so are the ports. If so, then it generates a event with a name of "Land_attack". It makes this check even if is false.

Returns: true if the connection is now hot (or was upon entry), false otherwise.

• check_hot (c: connection, state: count): bool checks the given connection against the various policy variables discussed above, and bumps the connection's hot field if it matches the policies for being sensitive, and does not match the various exceptions. It also uses check_spoof to see if the connection reflects a possible spoofing attack; and terminates the connection if terminate_successful_service indicates so.
  The caller indicates the connection's state in the second parameter to the function, using one of the values given in the Table below. As noted in the Table, the processing differs depending on the state.

Different connection states to use when calling check hot

State	Meaning	Tests
CONN_ATTEMPTED	Connection attempted, no reply seen. Note that you should also use this value for scans with undetermined state, such as possible stealth scans. For example, connection half_finished does this.	check_spoof
CONN_ESTABLISHED	Connection established. Also used for connections apparently established, per partial_connection.	check_spoof, flag_successful_service, flag_successful_inbound service, allow_services_to, terminate_successful_inbound_service
APPL_ESTABLISHED	The connection has reached application-layer establishment. For example, for Telnet or Rlogin, this is after the user has authenticated.	allow_services_to, allow_service_pairs, allow_pairs, allow_16_net_pairs, hot_srcs, hot_dsts, hot_src_24nets, hot_dst_24nets
CONN_FINISHED	The connection has finished, either cleanly or abnormally (for example, connection_reset.	Same as APPL_ESTABLISHED, if the connection exchanged non-zero amounts of data in both directions, and if the service wasn't one of the ones that generates APPL_ESTABLISHED
CONN_REJECTED	The connection attempt was rejected by the server.	check_spoof, flag_rejected_service

In general, the pattern is to make one call when the connection is first seen, either CONN_ATTEMPTED, CONN_ESTABLISHED, or CONN_REJECTED. If the application is one for which connections should only be considered "established" after a successful pre-exchange between originator and responder, then a subsequent call is made with a state of APPL_ESTABLISHED. The idea here is to provide a way to filter out what are in fact not really successful connections so that they are not analyzed in terms of successful service. Finally, for services that don't use APPL_ESTABLISHED, a call is made instead when the connection finishes for some reason, using state CONN_FINISHED.

Note: This approach delays noticing until the connection is over, which might be later than you want, in which case you may need to edit check_hot to provide the desired functionality.

Returns: true if the connection is now hot (or was upon entry), false otherwise.

The scan Analyzer

The scan analyzer detects connection attempts to numerous machines (address scanning), connection attempts to many different services on the same machine (port scanning), and attempts to access many different accounts (password guessing). The basic methodology is to use tables to keep track of the distinct addresses and ports to which a given host attempts to connect, and to trigger notices when either of these reaches a specified size.

Deficiency: As currently written, the analyzer will not detect distributed scans, i.e., when many sites are used to probe individually just a few, but together a large number, of ports or addresses.

A powerful technique that Bro potentially provides is dropping border connectivity with remote scanning sites, though you must supply the magic script to talk with your router and effect the block. See drop_address below for a discussion of the interface provided.

Note: Naturally, providing this capability means you might become vulnerable to denial-of-service attacks in which spoofed packets are used in an attempt to trigger a block of a site to which you want to have access.

scan variables

In addition to internal variables for its bookkeeping, the analyzer provides the following redefinable variables:

report_peer_scan : set[count] Generate an alarm whenever a remote host (as determined by is_local_address) has attempted to connect to the given number of distinct hosts.

Default: { 100, 1000, 10000, }. So, for example, if a remote host attempts to connect to 3,500 different local hosts, a report will be generated when it makes the 100th attempt, and another when it makes the 1,000th attempt.

• report_outbound_peer_scan : set[count] The same as report_peer_scan, except for connections initiated locally.

Default: { 1000, 10000, }.

• possible_port_scan_thresh : count Initially, port scan detection is done based on how many different ports a given host connects to, regardless of on which hosts. Once this threshold is reached, however, then the analyzer begins tracking ports accessed per-server, which is important for reducing false positives. Note: The reason this variable exists is because it is very expensive to track per-server ports accessed for every active host; this variable limits such tracking to only active hosts contacting a significant number of different ports. Default: 25.

• report_accounts_tried : set[count] Whenever a remote host has attempted to access a number of local accounts present in this set, generate an alarm. Each distinct username/password pair is considered a different access. Default: { 25, 100, 500, }.

• report_remote_accounts_tried : set[count] The same, except for access to remote accounts rather than local ones. Default: { 100, 500, }.

• skip_accounts_tried : set[addr] Do not do bookkeeping for account attempts for the given hosts. Default: empty.

• skip_outbound_services : set[port] Do not do outbound-scanning bookkeeping for connections involving the given services. Default: allow_services, ftp, addl_web (see next item).

• addl_web : set[port] Additional ports that should be considered as Web traffic (and hence skipped for outbound-scan bookkeeping). Default: { 81/tcp, 443/tcp, 8000/tcp, 8001/tcp, 8080/tcp, }.

• skip_scan_sources : set[addr] Hosts that are allowed to address-scan without complaint. Default: scooter.pa-x.dec.com, scooter2.av.pa-x.dec.com (AltaVista crawlers; you get the idea.)

• skip_scan_nets_24 : set[addr, port] /24 networks that are allowed to address scan for the given port without complaint. Default: empty.

• can_drop_connectivity : bool True if the Bro has the capability of dropping connectivity, per drop_address. Default: false.

• shut_down_scans : set[port] Scans of these ports trigger connectivity-dropping (if the Bro is capable of dropping connectivity), unless shut_down_all_scans is defined (next item). Default: empty.

• shut_down_all_scans : bool Ignore shut_down_scans and simply drop all scans regardless of service. Default: false.

• shut_down_thresh : count Shut down connectivity after a host has scanned this many addresses. Default: 100.

• never_shut_down : set[addr] Purported scans from these addresses are never shut down. Default: the root name servers (a.root-servers.net through m.root-servers.net).

scan functions

The standard scan script provides the following functions:

• drop_address (a: addr, msg: string) Drops external connectivity to the given address and generates a notification using the given message.

Dropping connectivity requires all of the following to be true:

• can_drop_connectivity is true. * The address is neither localnor a neighbor (See Site variables). * The address is not in never_shut_down.

If these checks succeed, then the script simply attempts to invoke a shell script drop-connectivity with a single argument, the IP address to block. It is up to you to provide the script, using whatever interface to your router/firewall you have available.

The function does not return a value.

• check_scan (c: connection, established: bool, reverse: bool): bool Updates the analyzer's internal bookkeeping on the basis of the new connection c. If established is true, then the connection was successfully established, otherwise not. If reverse is true, then the function should consider the originator/responder fields in the connection's record as reversed. Note: This last is needed for some unusual new connections that may reflect stealth scanning. For example, when the event engine sees a SYN-ack without a corresponding SYN, it instantiates a new connection with an assumption that the SYN-ack came from the responder (and it missed the initial SYN either due to split routing (See Split routing), a packet drop (See Split routing), or Bro having started running after the initial SYN was sent).

If the originating host's activity matches the policy defined by the variables above, then the analyzer logs this fact, and possibly attempts to drop connectivity to the originating host. The function also schedules an event for 24 hours in the future (or when Bro terminates) to generate a summary of the scanning activity (so if the host continues scanning, you get a report on how many hosts it wound up scanning).

Deficiency: This time interval should be selectable.

Note: Purported scans of the FTP data port (20/tcp) or the ident service (113/tcp) are never reported or dropped, as experience has shown they yield too many false hits.

The function does not return a value.

scan event handlers

The standard scan script defines one event handler:

• account_tried (c: connection, user: string, passwd: string) The given connection made an attempt to access the given username and password. Each distinct username/password pair is considered a new access. The event handler generates an alarm if the access matches the logging policy outlined above.

Note: account_tried events are generated by login and ftp analyzers.

The port-name Analysis Script

The port-name utility module provides one redefinable variable and one callable function:

• port_names : table[port] of string Maps TCP/UDP ports to names for the services associated with those ports. For example, 80/tcp maps to "http". These names are used by the conn analyzer when generating connection logs (See Generic Connection Analysis).

• endpoint_id (h: addr, p: port): string Returns a printable form of the given address/port connection endpoint. The format is either <address>/<service-name> or <address>/<port-number> depending on whether the port appears in port_names.

The brolite Analysis Script

The brolite module is intended to provide a convenient way to run (almost) all of the analyzers. It @load's the following other modules and analyzers: alarm, dns, hot, port-name, frag, tcp, scan, weird, finger, ident, ftp, login and portmapper. So you can run Bro using bro -i in0 brolite to have it analyze traffic on interface in0 using the above analyzers; or you can @load brolite to load in the above analyzers.

Note: The brolite analyzer doesn't load http (because it can prove a very high load for many sites) nor experimental analyzers such as stepping or backdoor.

The alarm Analysis Script

The alarm utility module redefines a single variable:

• bro_alarm_file : file A special Bro variable used internally to specify a file where Bro should record messages logged by alarm statements (as well as generating real-time notifications via syslog).

Default: if the $BRO_LOG_SUFFIX environment variable is defined, then alarm.@code{<$BRO_LOG_SUFFIX>}, otherwise alarm.log.

See bro_alarm_file for further discussion.

If you do not include this module, then Bro records alarm messages to stderr.

Here is a sample definition of alarm_hook:

global msg_count: table[string] of count &default = 0;

event alarm_summary(msg: string)
   {
   alarm fmt("(%s) %d times", msg, msg_count[msg]);
   }

function alarm_hook(msg: string): bool
   {
   if ( ++msg_count[msg] == 1 )
       # First time we've seen this message - log it.
       return T;

   if ( msg_count[msg] == 5 )
       # We've seen it five times, enough to be worth
       # summarizing.  Do so five minutes from now,
       # for whatever total we've seen by then.
       schedule +5 min { alarm_summary(msg) };

   return F;
   }

You can also control Bro's alarm processing by defining the special function alarm-hook. It takes a single argument, msg: string, the message in a just-executed alarm statement, and returns a boolean value: true if Bro should indeed log the message, false if not. The above example shows a definition of alarm_hook that checks each alarm message to see whether the same text has been logged before. It only logs the first instance of a message. If a message appears at least five times, then it schedules a future alarm_summary event for 5 minutes in the future; the purpose of this event is to summarize the total number of times the message has appeared at that point in time.

The active Analysis Script

The active utility module provides a single, non-redefinable variable that holds information about active connections:

• active_conn : table[conn_id] of connection Indexed by a conn_id giving the originator/responder addresses/ports, returns the connection's connection record. As usual, accessing the table with a non-existing index results in a run-time error, so you should first test for the presence of the index using the in operator.

Default: empty.

This functionality is quite similar to that of the active_connection function, and Deficiency:arguably this module should be removed in favor of the function. It does, however, provide a useful example of maintaining bookkeeping by defining additional handlers for events that already have handlers elsewhere.

The demux Analysis Script

The demux utility module provides a single function:

• demux_conn (id: conn_id, tag: string, otag: string, rtag: string): bool Instructs Bro to write (``demultiplex) the contents of the connection with the given id to a pair of files whose names are constructed out of tag, otag, and rtag, as follows.

The originator-to-responder direction of the connection goes into a file named:

<otag >.<tag >.<orig-addr >.<orig-port >-<resp-addr >.<resp-port >

and the other direction in:

<rtag >.<tag >.<resp-addr >.<resp-port >-<orig-addr >.<orig-port >

Accordingly, tag can be used to associate a unique label with the pair of files, while otag and rtag provide distinct labels for the two directions.

If Bro is already demuxing the connection, or if the connection is not active, then nothing happens, and the function returns false. Otherwise, it returns true.

Bro places demuxed streams in a directory defined by the redefinable global demux_dir, which defaults in the usual fashion to open_log_file("xscript").

Deficiency:Experience has shown that it would be highly convenient if Bro would demultiplex the entire connection contents into the files, instead of just the part of the connection seen subsequently after the call to demux_conn. One way to do this would be for demux_conn to offset the contents in the file by the current stream position, and then to invoke a utility tool that goes through the Bro output trace file and copies the contents up to the current stream position to the front of the file. This utility tool might even be another instance of Bro running with suitable arguments.}

The dns Analysis Script

The dns module deals with Bro's internal mapping of hostnames to/from IP addresses. Deficiency: There is no DNS protocol analyzer available at present. Furthermore, Deficiency: the lookup mechanisms discussed here are not available to the Bro script writer, other than implicitly by using hostnames in lieu of addresses in variable initializations (see [[Reference Manual: #Hostnames vs addresses).|Hostnames vs addresses).]]

The module's function is to handle different events that can occur when Bro resolves hostnames upon startup. Bro maintains its own cache of DNS information which persists across invocations of Bro on the same machine and by the same user. The role of the cache is to allow Bro to resolve hostnames even in the face of DNS outages; the philosophy is that it's better to use old addresses than none at all, and this helps harden Bro against attacks in which the attacker causes DNS outages in order to prevent Bro from resolving particular sensitive hostnames (e.g., hot_srcs ). The cache is stored in the file ``.bro-dns-cache in the user's home directory. You can delete this file whenever you want, for example to purge out old entries no longer needed, and Bro will recreate it next time it's invoked using -P.

Currently, all of the event handlers are invoked upon comparing the results of a new attempt to look up a name or an address versus the results obtained the last time Bro did the lookup. When Bro looks up a name for the first time, no events are generated.

Also, Bro currently only looks up hostnames to map them to addresses. It does not perform inverse lookups.

The dns_mapping record

All of the events handled by the module include at least one record of DNS mapping information, defined by the dns_mapping type shown in the example below. The corresponding fields are:

• creation_time When the mapping was created.

• req_host The hostname looked up, or an empty string if this was not a hostname lookup.

• req_addr The address looked up (reverse lookup), or 0.0.0.0 if this was not an address lookup.

• valid True if an answer was received for a lookup (even if the answer was that the request name or address does not exist in the DNS).

• hostname The hostname answer in response to an address lookup, or the string "@code{<none>"} if an answer was received but it indicated there was no PTR record for the given address.

• addrs A set of addresses in response to a hostname lookup. Empty if an answer was received but it indicated that there was no A record for the given hostname.

type dns_mapping: record {
   creation_time: time;  # When the mapping was created.

   req_host: string;     # The hostname in the request, if any.
   req_addr: addr;       # The address in the request, if any.

   valid: bool;          # Whether we received an answer.
   hostname: string;     # The hostname in the answer, or "<none>".
   addrs: set[addr];     # The addresses in the answer, if any.
};

dns variables

The modules provides one redefinable variable:

• dns_interesting_changes : set[string] The different DNS events have names associated with them. If the name is present in this set, then the event will generate a notice, otherwise not.

One exception to this list is that DNS changes involving the loopback address 127.0.0.1 are always considered notice-worthy, since they may reflect DNS corruption.

Default: { "unverified", "old name", "new name", "mapping", }.

dns event handlers

The DNS module supplies the following event handlers:

• dns_mapping_valid (dm: dns_mapping) The given request was looked up and it was identical to its previous mapping.

• dns_mapping_unverified (dm: dns_mapping) The given request was looked up but no answer came back.

• dns_mapping_new_name (dm: dns_mapping) In the past, the given address did not resolve to a hostname; this time, it did.

• dns_mapping_lost_name (dm: dns_mapping) In the past, the given address resolved to a hostname; now, that name has gone away. (An answer was received, but it stated that there is no hostname corresponding to the given address.)

• dns_mapping_name_changed (old_dm: dns_mapping, new_dm: dns_mapping) The name returned this time for the given address differs from the name returned in the past.

• dns_mapping_altered (dm: dns_mapping, old_addrs: set[addr], new_addrs: set[addr]) The addresses associated with the given hostname have changed. Those in old_addrs used to be part of the set returned for the name, but aren't any more; while those in new_addrs didn't used to be, but now are. There may also be some unchanged addresses, which are those in dm$addrs but not in new_addrs.

The finger Analyzer

The finger analyzer processes traffic associated with the Finger service RFC-1288. Bro instantiates a finger analyzer for any connection with service port 79/tcp (if you @load the finger analyzer in your script, or define your own finger_request or finger_reply handlers, of course).

The analyzer uses a capture filter of ``port finger (See: Filtering).

In the past, attackers often used Finger requests to obtain information about a site's users, and sometimes to launch attacks of various forms (buffer overflows, in particular). In our experience, exploitation of the service has greatly diminished over the past years (no doubt in part to the service being increasingly turned off, or prohibited by firewalls). Now it is only rarely associated with an attack.

finger variables

The standard script defines two redefinable variables:

• hot_names : set[string] A list of usernames that should be considered sensitive (notice-worthy) if included in a Finger request.

Default: { "root", "lp", "uucp", "nuucp", "demos", "operator", "sync", "guest", "visitor", }.

• max_request_length : count The largest reasonable request size (used to flag possible buffer overflow attacks). Bro marks a connection as ``hot if its request exceeds this length, and truncates its logging of the request to this many bytes, followed by "...".

Default: 80.

finger event handlers

The standard script defines one event handler:

• finger_request (c: connection, request: string, full: bool) Invoked upon connection c having made the request request. The full flag is true if the request included the ``long format option (which the event engine will have removed from the request).

The standard script flags long requests and truncates them as noted above, and then checks whether the request is for a name in hot_names. It then formats the request either by placing double quotation marks around it, or, if the request was empty---indicating a request for information on all users---the request is changed to the string ALL with no quotes around it.

If the originator already made a request, then this additional request is placed in parentheses (though multiple requests violate the Finger protocol). If the request was for the full format, then the text ``(/W) is appended to the request. Finally, the request is appended to the connection's field.

The event engine generates an additional event that the predefined finger script does not handle:

• finger_reply (c: connection, reply_line: string) Generated for each line of text sent in response to the originator's request.

The frag Analysis Script

The frag utility module simply refines the capture filter (See: Filtering) so that Bro will capture and reassemble IP fragments. Bro reassembles any fragments it receives; but normally it doesn't receive any, except the beginnings of TCP fragments (see the tcp module), and UDP port 111 (per the portmapper module).

So, to make Bro do fragment reassembly, you simply use ``load frag. It effects this by adding:

   (ip[6:2] & 0x3fff != 0) and tcp

to the filter. The first part of this expression matches all IP fragments, while the second restricts those matched to TCP traffic. We would like to use:

   (ip[6:2] & 0x3fff != 0) and (tcp or udp port 111)

to also include portmapper fragments, but that won't work---the port numbers will only be present in the first fragment, so the packet filter won't recognize the subsequent fragments as belonging to a UDP port 111 packet, and will fail to capture them.

Note: Alternatively, we might be tempted to use ``(tcp or udp) and so capture all UDP fragments, including port 111. This would work in principle, but in practice can capture very high volumes of traffic due to NFS traffic, which can send all of its file data in UDP fragments.}

The hot-ids Analysis Script

The hot-ids module defines a number of redefinable variables that specify usernames Bro should consider sensitive:

• forbidden_ids set[string] lists usernames that should never be used. If Bro detects use of one, it will attempt to terminate the corresponding connection.

Default: { "uucp", "daemon", "rewt", "nuucp", "EZsetup", "OutOfBox", "4Dgifts", "ezsetup", "outofbox", "4dgifts", "sgiweb", }. All of these correspond to accounts that some systems have enabled by default (with well-known passwords), except for "rewt", which corresponds to a username often used by (weenie) attackers.

Deficiency: The repeated definitions such as "EZsetup" and "ezsetup" reflect that this variable is a set and not a pattern. Consequently, the exact username must appear in it (with a pattern, we could use character classes to match both upper and lower case).

• forbidden_ids_if_no_password : set[string] Same as forbidden_ids except only considered forbidden if the login succeeded with an empty password.

Default: "lp", a default passwordless IRIX account.

• forbidden_id_patterns : pattern A pattern giving user ids that should be considered forbidden. Deficiency: This pattern is currently only used to check Telnet/Rlogin user ids, not ids seen in other contexts, such as FTP sessions.

Default: /(y[o0]u)(r|ar[e3])([o0]wn.*)/, a particularly egregious style of username of which we've observed variants in different break-ins.

• always_hot_ids : set[string] A list of usernames that should always be considered sensitive, though not necessarily so sensitive that they should be terminated whenever used.

Default: { "lp", "warez", "demos", forbidden_ids, }. The "lp" and "demos" accounts are specified here rather than forbidden_ids because it's possible that they might be used for legitimate accounts. "warez" (for ``wares, i.e., bootlegged software) is listed because its use likely constitutes a policy violation, not a security violation.

Note: forbidden_ids is incorporated into always_hot_ids to avoid replicating the list of particularly sensitive ids by listing it twice and risking inconsistencies.

• hot_ids set[string] User ids that generate notices if the user logs in successfully.

Default: { "root", "system", always_hot_ids, }. The ones included in addition to always_hot_ids are only considered sensitive if the user logs in successfully.

The ftp Analyzer

The ftp analyzer processes traffic associated with the FTP file transfer service RFC-959. Bro instantiates an ftp analyzer for any connection with service port 21/tcp, providing you have loaded the ftp analyzer, or defined a handler for ftp_request or ftp_reply.

The analyzer uses a capture filter of ``port ftp (See: Filtering). It generates summaries of FTP sessions; looks for sensitive usernames, access to sensitive files, and possible FTP ``bounce attacks, in which the host specified in a ``PORT or ``PASV directive does not correspond to the host sending the directive; or in which a different host than the server (client) connects to the endpoint specified in a PORT (PASV) directive.

The ftp_session_info record

The main data structure managed by the ftp analyzer is a collection of ftp_session_info records, where the record type is shown below:

type ftp_session_info: record {
   id: count;              # unique number associated w/ session
   user: string;           # username, if determined
   request: string;        # pending request or requests
   num_requests: count;    # count of pending requests
   request_t: time;        # time of request
   log_if_not_denied: bool;        # unless code 530 on reply, log it
   log_if_not_unavail: bool;       # unless code 550 on reply, log it
   log_it: bool;           # if true, log the request(s)
};

The corresponding fields are:

• id The unique session identifier assigned to this session. Sessions are numbered starting at 1 and incrementing with each new session.

• user The username associated with this session (from the initial FTP authentication dialog), or an empty string if not yet determined.

• request The pending request, if the client has issued any. Ordinarily there would be at most one pending request, but a client can in fact send multiple requests to the server all at once, and an attacker could do so attempting to confuse the analyzer into mismatching responses with requests, or simply forgetting about previous requests.

• num_requests A count of how many requests are currently pending.

• request_t The time at which the pending request was issued.

• log_if_not_denied If true, then when the reply to the current request comes in, Bro should log it, unless the reply code is 530 (``denied).

• log_if_not_unavail If true, then when the reply to the current request comes in, Bro should log it, unless the reply code is 550 (``unavail).

• log_it If true, then when the reply to the current request comes in, Bro should log it.

ftp variables

The standard script defines the following redefinable variables:

• ftp_guest_ids : set[string] A set of usernames associated with publicly accessible ``guest services. Bro interprets guest usernames as indicating Bro should use the authentication password as the effective username.

Default: { "anonymous", "ftp", "guest", }.

• ftp_skip_hot : set[addr, addr, string] Entries indicate that a connection from the first given address to the second given address, using the given string username, should not be treated as hot even if the username is sensitive.

Default: empty.

Example: redefining ftp_skip_hot using

redef ftp_skip_hot: set[addr, addr, string] += {
   [[bob1.dsl.home.net, bob2.dsl.home.net], 
     bob.work.com, "root"], };

would result in Bro not noticing FTP connections as user "root" from either bob1.dsl.home.net or bob2.dsl.home.net to the server running on bob.work.com.

• ftp_hot_files : pattern Bro matches the argument given in each FTP file manipulation request (RETR, STOR, etc.) against this pattern to see if the file is sensitive. If so, and if the request succeeds, then the access is logged.

Default: aggdrop a pattern that matches various flavors of password files, plus any string with eggdrop in it. @emph{Note: Eggdrop is an IRC management tool often installed by certain attackers upon a successful break-in.}

• ftp_not_actually_hot_files : pattern A pattern giving exceptions to ftp_hot_files. It turns out that a pattern like /passwd/ generates a lot of false hits, such as from passwd.c (source for the passwd utility; this can turn up in FTP sessions that fetch entire sets of utility sources using MGET) or passwd.html (a Web page explaining how to enter a password for accessing a particular page).

Default: /(passwd|shadow).*@code{.(c|gif|htm|pl|rpm|tar|zip)/} .

• ftp_hot_guest_files pattern Files that guests should not attempt to access.

Default: .rhosts and .forward .

• skip_unexpected : set[addr] If a new host (address) unexpectedly connects to the endpoint specified in a PORT or PASV directive, then if either the original host or the new host is in this set, no message is generated. The idea is that you can specify multi-homed hosts that frequently show up in your FTP traffic, as these can generate innocuous warnings about connections from unexpected hosts.

Default: some hp.com hosts, as an example. Most are specified as raw IP addresses rather than hostnames, since the hostnames don't always consistently resolve.

• skip_unexpected_net : set[addr] The same as skip_unexpected, except addresses are masked to /24 and /16 before looked up in this set.

Default: empty.

In addition, ftp_log holds the name of the FTP log file to which Bro writes FTP session summaries. It defaults to open_log_file("ftp").

Here is an example of what entries in this file look like:

972499885.784104 #26 131.243.70.68/1899 > 64.55.26.206/ftp start
972499886.685046 #26 response (220 tuvok.ooc.com FTP server
   (Version wu-2.6.0(1) Fri Jun 23 09:17:44 EDT 2000) ready.)
972499886.686025 #26 USER anonymous/IEUser@ (logged in)
972499887.850621 #26 TYPE I (ok)
972499888.421741 #26 PASV (227 64.55.26.206/2427)
972499889.493020 #26 SIZE /pub/OB/4.0/JOB-4.0.3.zip (213 1675597)
972499890.135706 #26 *RETR /pub/OB/4.0/JOB-4.0.3.zip, ABOR (complete)
972500055.491045 #26 response (225 ABOR command successful.)

Here we see a transcript of the 26th FTP session seen since Bro started running. The first line gives its start time and the participating hosts and ports. The next line (split across two lines above for clarity) gives the server's welcome banner. The client then logged in as user ``anonymous, and because this is one of the guest usernames, Bro recorded their password too, which in this case was ``IEUser (a useless string supplied by their Web browser). The server accepted this authentication, so the status on the line is ``(logged in).

The client then issues a request for the Image file type, to which the server agreed. Next they issued a PASV directive, and received a response instructing them to connect to the server on port 2427/tcp for the next transfer. At this point, after issuing a SIZE directive (to which the server returned 1,675,597 bytes), they send RETR to fetch the file /pub/OB/4.0/JOB-4.0.3.zip. However, before the transfer completed, they issued ABOR, but the transfer finished before the server processed the abort, so the log shows a status of completed. Furthermore, because the client issued two commands without waiting for an intervening response, these are shown together in the log file, and the line marked with a ``* so it draws the eye. Finally, because Bro paired up the (completed) with the multi-request line, it then treats the response to the ABOR command as a reply by itself, showing in the last line that the server reported it successfully carried out the abort.

The corresponding lines in the @file{conn} file look like:

   972499885.784104 565.836 ftp 118 427 131.243.70.68 64.55.26.206
       RSTO L #26 anonymous/IEUser@
   972499888.984116 165.098 ftp-data ? 1675597 131.243.70.68 

64.55.26.206 RSTO L

The first line summarizes the FTP control session (over which the client sends its requests and receives the server's responses). It includes an addl annotation of ``#26 anonymous/IEUser, summarizing the session number (so you can find the corresponding records in the ftp log file) and the authentication information.

The second line summarizes the single FTP data transfer, of 1,675,597 bytes. The amount of data sent by the client for this connection is shown as unknown because the client aborted the connection with a RST (hence the state RSTO). For connections that Bro does not look inside (such as FTP data transfers), it learns the amount of data transferred from the sequence numbers of the SYN and FIN connection control packets, and can't (reliably) learn them for the sender of a RST. (It can for the receiver of the RST.)

They also aborted the control session (again, state RSTO), but in this case, Bro captured all of the packets of the session, so it could still assign sizes to both directions.

ftp functions

The standard ftp script provides one function for external use:

• is_ftp_data_conn (c: connection): bool Returns true if the given connection matches one we're expecting as the data connection half of an FTP session. @emph{Note: This function is not idempotent: if the connection matches an expected one, then Bro updates its state such that that connection is no longer expected. It also logs a discrepancy if the connection appears to be usurping another one that generated either a ``PORT or a ``PASV directive.}

Also returns true if the source port is 20/tcp and there's currently an FTP session active between the originator and responder, in case for some reason Bro's bookkeeping is inconsistent.

ftp event handlers

The standard script handles the following events:

• ftp_request (c: connection, command: string, arg: string) Invoked upon the client side of connection c having made the request command with the argument arg.

The processing depends on the particular command:

• USER Specifies the username that the client wishes to use for authentication. If it is sensitive---in hot_ids (which the ftp analyzer accesses via a @load of hot-ids)---then the analyzer flags the FTP session as notice-worthy. In addition, if the username is in forbidden_ids, then the analyzer terminates the session.

The analyzer also updates the connection's addl field with the username.

• PASS Specifies the password to use for authentication.

If the password is empty and the username appears in forbidden_ids_if_no_password (also from the hot-ids analyzer), then the analyzer terminates the connection.

If the username corresponds to a guest account (ftp_guest_ids), then the analyzer updates the connection's addl field with the password as additional account information. Otherwise, it generates an account_tried event to facilitate detection of password guessing.

• PORT Instructs the FTP server to connect to the given IP address and port for delivery of the next FTP data item. The analyzer first checks the address/port specifier for validity. If valid, it will generate a notice if either the address specified in the directive does not match that of the client, or if the port corresponds to a ``privileged port, i.e., one in the range 0--1023. Finally, it establishes state so that is_ftp_data_conn can identify a subsequent connection corresponding to this directive as belonging to this FTP session.

• ACCT Specifies additional accounting information associated with a session, which the analyzer simply adds to the connection's field.

• APPE, CWD, DELE, MKD, RETR, RMD, RNFR, RNTO, STOR, STOU All of these manipulate files (and directories). The analyzer checks the filename against the policies to see if it is sensitive in the context of the given username (i.e., guest or non-guest), and, if so, marks the connection to generate a notice unless the operation fails. The analyzer also checks for an excessively long filename, currently by checking its length against a Deficiency:hardwired maximum of 250 bytes.

• ftp_reply (c: connection, code: count, msg: string, cont_resp: bool) Invoked upon the server side of connection c having replied to a request using the given status code and text message. cont_resp is true if the reply line is tagged as being continued to the next line. The analyzer only processes requests when the last line of a continued reply is received.

The analyzer checks the reply against any expected for the connection (for example, ``log_if_not_denied) and generates notices accordingly. If the reply corresponds to a PASV directive, then it parses the address/port specification in the reply and generates notices in an analogous fashion as done by the ftp_request handler for PORT directives.

Finally, if the reply is not one that the analyzer is hardwired to skip (code 150, used at the beginning of a data transfer, and code 331, used to prompt for a password), then it writes a summary of the request and reply to the FTP log file (See: ftp variables). Also, if the reply is an ``orphan (there was no corresponding request, perhaps because Bro started up after the request was made), then the reply is summarized in the log file by itself.

The standard ftp script defines one other handler, an instance of

used to flush FTP session information

in case the session terminates abnormally and no reply is seen to the pending request(s).

ftp notices

The FTP analyzer can generate the following Notices:

• FTP::FTP_BadPort - Bad format in PORT/PASV * FTP::FTP_ExcessiveFilename - Very long filename seen * FTP::FTP_PrivPort - Privileged port used in PORT/PASV * FTP::FTP_Sensitive -Sensitive connection (as defined in hot) * FTP::FTP_UnexpectedConn - Data transfer from unexpected src. Suppose there's an FTP session between client A and server B, and either A issues a PORT or B issues a PASV. Then what's expected is that A will rendezvous with B using the port specified in the PORT/PASV. If instead a new IP address C connects to (or accepts from) the negotiated port, that generated FTP_UnexpectedConn.

The http Analyzer

The http analyzer processes traffic associated with the Hyper Text Transfer Protocol (HTTP) [RFC-1945], the main protocol used by the Web. Bro instantiates an http analyzer for any connection with service port 80/tcp, providing you have loaded the http analyzer, or defined a handler for http_request. It also instantiates an analyzer for service ports 8080/tcp and 8000/tcp, as these are often also used for Web servers.

The analyzer uses a capture filter of ``tcp dst port 80 or tcp dst port 8080 or tcp dst port 8000 (See: Filtering). Note: This filter excludes traffic sent by an HTTP server (that would be matched by tcp src port 80, etc.), because Deficiency: Bro doesn't yet have an analyzer for HTTP replies. It generates summaries of HTTP sessions (connections between the same client and server) and looks for access to sensitive URIs (effectively, URLs).

http variables

• sensitive_URIs : pattern Any HTTP method (e.g., GET, HEAD, POST) specifying a URI that matches this pattern is flagged as sensitive.

Default: URIs with /etc/passwd or /etc/shadow embedded in them, or /cfdocs/expeval (used in some Cold Fusion exploits). Note: This latter generates some false hits; it's mainly included just to convey the notion of looking for direct attacks rather than attacks used to exploit sensitive files like the first ones.

Deficiency: It would be very handy to have variables providing hooks for more context when considering whether a particular access is sensitive, such as whether the request was inbound or outbound.

• sensitive_post_URIs : pattern Any POST method specifying a URI that matches this pattern is flagged as sensitive.

Default: URIs with wwwroot embedded in them.

In addition, http_log holds the name of the HTTP log file to which Bro writes HTTP session summaries. It defaults to open_log_file("http").

Here we show an example of what entries in this file look like:

972482763.371224 %1596 start 200.241.229.80 > 131.243.2.12
%1596 GET /ITG.hm.pg.docs/dissect/portuguese/dissect.html
%1596 GET /vfrog/bottom.icon.gif
%1596 GET /vfrog/top.icon.gif
%1596 GET /vfrog/movies/off.gif
%1596 GET /vfrog/new.frog.small.gif

Here we see a transcript of the 1596th HTTP session seen since Bro started running. The first line gives its start time and the participating hosts. The next five lines all correspond to GET methods retrieving different items from the Web server.

Deficiency: Bro can't log whether the retrievals succeeded or failed because it doesn't yet have an HTTP reply analyzer.

The corresponding lines in the conn file look like:

   972482762.872695 481.551 http 441 5040 131.243.2.12 200.241.229.80
       S3 X %10596
   972482764.686470 18.7611 http 596 7712 131.243.2.12 200.241.229.80
       S3 X %10596
   972482764.685047 ? http 603 2959 131.243.2.12 200.241.229.80
       S1 X %10596

That there are three rather than five reflects (i) that the client used persistent HTTP, and so didn't need one connection per item, but also (ii) the client used three parallel connections (the maximum the standard allows is only two) to fetch the items more quickly. As with FTP sessions, the %10596 addl annotation lets you correlate the conn entries with the log entries.

Note: All three of the connections wound up in unusual states. The first two are in state S3, which, as indicated by Table 7.3, means that the responder (in this case, the Web server) attempted to close the connection, but their was no reply from the originator. The last is in state S1, indicating that neither side attempted to close the connection (which is why no duration is listed for the connection).

http event handlers

The standard HTTP script defines one event handler:

• http_request c: connection, request: string, URI: string Invoked whenever the client side of the given connection generates an HTTP request. request gives the HTTP method and URI the associated resource. The analyzer matches the URI against the ones defined as sensitive, as given above.

Deficiency: As mentioned above, the event engine does not currently generate an http_reply event. This is for two reasons: first, the HTTP request stream is much lower volume than the HTTP reply stream, and I was interested in the degree to which Bro could get away without analyzing the higher volume stream. (Of course, this argument is shallow, since one could control whether or not Bro should analyze HTTP replies by deciding whether or not to define an http_reply handler.) Second, matching HTTP replies in their full generality involves a lot of work, because the HTTP standard allows replies to be delimited in a number of ways. That said, most of the work for implementing http_reply is already done in the event engine, but it is missing testing and debugging.

The ident Analyzer

The ident analyzer processes traffic associated with the Identification Protocol [RFC-1413], which provides a simple service whereby clients can query Ident servers to discover user information associated with an existing connection between the server's host and the client's host. Bro instantiates an ident analyzer for any connection with service port 113/tcp, providing you have loaded the ident analyzer, or defined a handler for ident_request, ident_reply, or ident_error.

The analyzer uses a capture filter of ``tcp port 113 (See: Filtering). The ident_reply handler annotates the addl field of the connection for which the Ident client made its query with the user information returned in the reply. It also checks the user information against sensitive usernames, because a match indicates that the connection in the Ident query was initiated by a possibly-compromised account.

ident variables

The standard script defines the following pair of redefinable variables:

• hot_ident_ids : set[string] usernames to flag as sensitive if they appear in an Ident reply.

Default: always_hot_ids (See: hot-ids Analysis Script).

• hot_ident_exceptions : set[string] usernames not to consider sensitive even if they appear in hot_ident_ids.

Default: { "uucp", "nuucp", "daemon", }. These usernames are exceptions because daemons sometimes run with the given user ids and their use is often innocuous.

ident event handlers

The standard script handles the following events:

• ident_request (c: connection, lport: port, rport: port) Invoked when a client request arrives on connection c, querying about the connection from local port lport to remote port rport, where local and remote are relative to the client.

• ident_reply (c: connection, lport: port, rport: port, user_id: string, system: string) Invoked when a server replies to an Ident request. lport and rport are again the local and remote ports (relative to the client) of the connection being asked about. user_id is the user information returned in the Ident server's reply, and system is information regarding the operating system (the Ident specification does not further standardize this information).

The handler annotates the queried connection with the user information, which it also checks against hot_ident_ids and hot_ident_exceptions as discussed above. At present, it does nothing with the system information.

• ident_error (c: connection, lport: port, rport: port, line: string) Invoked when the given request yielded an error reply from the Ident server. The handler annotates the connection with ident/<error>, where error is the text given in line.

The irc Analyzer

The IRC analyzer processes traffic from chat sessions that use the IRC (Internet Relay Chat) protocol. It can analyze client-server connections and server-server connections.

Bro instantiates an IRC analyzer for any connection with service ports 6666/tcp or 6667/tcp, providing you have loaded the IRC analyzer, or defined a handler for one of the IRC events. It it also possible to analyze server connections, but to do so you need to recompile Bro to include the necessary ports if they are not the usual ones.

Bro can analyze compressed connections if it sees the beginning of the connection.

irc records

The standard script defines a record for users and one for channels. This is the user record:

type irc_user: record {
	u_nick: string;			# nick name
	u_real: string;			# real name
	u_host: string;			# client host
	u_channels: set[string];	# channels user is a member of
	u_is_operator: bool;		# user is server operator
	u_conn: connection;
}

This record represents a user inside the IRC network. The corresponding fields are:

• u_nick The nick name of the user.

• u_real The real name of the user.

• u_host This is the client's host name.

• u_channels A list of channels the user has joined.

• u_isOp If the user got operator status in the IRC network this will be set to true.

• u_conn The TCP connection which this IRC connection is based on.

This is the channel record:

type irc_channel: record {
	c_name: string;		# channel name
	c_users: set[string];	# users in channel
	c_ops: set[string];	# channel operators
	c_type: string;		# channel type
	c_modes: string;	# channel modes
}

This record represents a channel inside the IRC network. The corresponding fields are:

• c_name The name of the channel.

• c_users A list of nick names of users in this channel.

• c_ops A list of nick names of users with operator status in this channel.

• c_type The channel type.

• c_modes The channel modes.

irc variables

The standard script defines the following set of redefinable variables:

• IRC::hot_words list of regular expressions which will generate notice messages. The analyzer searches for these patterns in user messages, notices and all unknown IRC commands.

• IRC::ignore_in_other_msgs: set[string] list of IRC commands which are ignored in the events for unknown commands.

• IRC::ignore_in_other_responses: set[count] list of IRC return codes which are ignored in the event for unknown return codes.

These variables contain information about users and channels which were identified by Bro.

• IRC::users: table[string] contains all identified IRC users as irc_user objects.

• IRC::channels: table[string] contains all identified IRC channels as irc_channel objects.

irc event handlers

The standard script handles the following events:

• irc_privmsg_message (c: connection, source: string, target: string, message: string) A user sent a message to another user or channel.

IRC command: PRIVMSG

The source is the user who sent the message to the target user/channel. Message contains the data sent to the target.

• irc_notice_message (c: connection, source: string, target: string, message: string) This is very similar to the irc_privmsg_message. It is typically used by services or client scripts to send status messages.

IRC command: NOTICE

The source is the user who sent the message to the target user/channel. Message contains the data sent to the target.

• irc_squery_message (c: connection, source: string, target: string, message: string) This event is activated if somebody sends a message to an IRC service.

IRC command: SQUERY

The source is the user who sent the message to the target service. Message contains the data sent to the target.

• irc_enter_message (c: connection, nick: string, realname: string) Every time a user enters the IRC network this event occurs.

IRC command: USER

Nick contains the selected nick name of the user and realname the user's name in real life.

• irc_quit_message (c: connection, nick: string, message: string) Every time a user quits the IRC network this event occurs.

IRC command: QUIT

Nick contains the nick name of the sender. An optional quit message is included in message.

• irc_join_message (c: connection, infoList: irc_join_list) If a user joins one or more IRC channels this event occurs.

IRC command: JOIN

The infoList contains a list of joined channel names and - if provided by user - the passwords for them.

• irc_part_message (c: connection, nick: string, channels: string_set, message: string) If a user exits one or more IRC channels this event occurs.

IRC command: PART

Nick contains the nick name of the user. Channels is a set of channel names. If the user supplies a quit message it is included in message.

• irc_nick_message (c: connection, who: string, newnick: string) This event occurs when users change their nick names.

IRC command: NICK

Who contains the IRC message prefix which includes the user nick and host. Newnick is the new nick name of this user.

• irc_invalid_nick (c: connection) This event occurs when users change their nick names and the name was invalid.

IRC response to: NICK

• irc_network_info (c: connection, users: count, services: count, servers: count) This a summary of the status of the whole IRC network.

IRC response to: LUSERS

Users, services and servers are the total number of users, services and IRC servers connected to the IRC network.

• irc_server_info (c: connection, users: count, services: count, servers: count) This a summary of an IRC server status.

IRC response to: LUSERS

Users, services and servers are the total number of users, services and IRC servers connected with this IRC server.

• irc_channel_info (c: connection, channels: count) Displays the total number of channels.

IRC response to: LUSERS

Channels is the number of IRC channels formed on this server (local + global).

• irc_who_message (c: connection, mask: string, oper: bool) The event occurs if an IRC user sent the WHO command to get information about an IRC user or channel.

IRC command: WHO

Mask is the target of the search. This can be a channel or user name, wildcards are allowed. If oper is true then the user asks only for operator user results.

• @code{irc_who_line (c: connection, target_nick: string, channel: string, user: string, host: string, server: string, nick: string, params: string, hops: count, realname: string)}

This includes several information about an IRC user.

IRC response to: WHO

Target_nick is the nick name of the IRC user who sent the WHO request. The username of the returned IRC user is included in user, his nick name in nick and real name in realname. The client DNS/IP address is host. Params includes the channel parameters for this user (e.g. "@" for channel operators). The user is connected to IRC server server and the number of servers between him and the requester is hops. Channel includes the channel name which was target for the request.

• irc_whois_message (c: connection, server: string, users: string) The event occurs if an IRC user sent the WHOIS command to get information about one or more IRC users.

IRC command: WHOIS

If server is given then the user wants this specific server to answer. Users is comma separated list of nick names for which information is requested.

• irc_whois_user_line (c: connection, nick: string, user: string, host: string, realName: string) This includes several information about an IRC user.

IRC response to: WHOIS

The user with nick name nick has the user name user and his real name is realname. The IRC client runs on host.

• irc_whois_operator_line (c: connection, nick: string) This response to an WHOIS command gives information if an IRC user is operator.

IRC response to: WHOIS

The IRC user with nick name nick has operator status.

• irc_whois_channel_line (c: connection, nick: string, channels: string_set) This response to an WHOIS command gives information on the channels of an IRC user.

IRC response to: WHOIS

The IRC user with nick name nick is member in all IRC channels of the variable channels.

• irc_oper_message (c: connection, user: string, password: string) This means that an IRC user requested operator status.

IRC command: OPER

The user and password parameters are used to authenticate the possible operator. They must fit to the IRCD server settings.

• irc_oper_response (c: connection, got_oper: bool) This is the answer to an operator request.

IRC response to: OPER

If the IRC user got operator status the got_oper variable is true.

• irc_kick_message (c: connection, prefix: string, channels: string, users: string, comment: string) An user requested to remove somebody from a channel.

IRC command: KICK

Prefix includes the requesters nick name and host. The user requested to remove the users (comma separated list) from the channels (comma separated list). If the requester provided an optional kick message it is included in comment.

• irc_error_message (c: connection, prefix: string, message: string) An IRC server sent an error message to one or more clients.

IRC command: ERROR

Prefix includes the server name and message contains the error message.

• irc_invite_message (c: connection, prefix: string, nickname: string, channel: string) An IRC user sent an invitation for a closed channel to another user.

IRC command: INVITE

Prefix includes the senders nick and host. The IRC user with the nick name nickname is invited to the channel with name channel.

• irc_mode_message (c: connection, prefix: string, params: string) An IRC user sent an user or channel mode message.

IRC command: MODE

• irc_squit_message (c: connection, prefix: string, server: string, message: string) This means that the disconnection of a server link was requested. This command is only available to operators.

IRC command: SQUIT

Prefix includes the requesters nick and host. Server is the host name of the server to disconnect and message contains an optional comment.

• irc_names_info (c: connection, c_type: string, channel: string, users: string_set) This reply to a NAMES command gives information what users are on what channels.

IRC response to: NAMES

C_type is "@" for secret, "*" for private and "=" for public channels. Channel contains the channel name. Users is a list of nick names that are member of this channel.

• @code{irc_dcc_message (c: connection, prefix: string, target: string, dcc_type: string, argument: string, address: addr, dest_port: count, size: count)}

An user sent a DCC request to another user to setup a direct connection between these users.

IRC command: PRIVMSG DCC

Prefix contains the requesters nick and host. Target contains the target user's nick name. Dcc_type can be "CHAT" for chat connections or "SEND" for file transfers. Argument contains the file name for file transfers or "chat" for chat connections. Address and dest_port specify where the target user should connect. Size is only given for file transfers and contains the file size in bytes.

• irc_request (c: connection, prefix: string, command: string, arguments: string) All client messages that do not fit to the other events are handled here.

Prefix is usually formated like this: <nickname>!<user>@<hostname>. Command contains the command string which was sent and arguments the corresponding argument values.

• irc_message (c: connection, prefix: string, command: string, message: string) All server messages that do not fit to the other events are handled here.

Prefix is usually the server name. Command contains the command string which was sent and message contains additional parameters.

• irc_response (c: connection, prefix: string, code: count, params: string) All server response messages that do not fit to the other events are handled here.

Prefix is usually the server name. Code is the numeric reply code and params contains any additional parameters.

The login Analyzer

The login analyzer inspects interactive login sessions to extract username and password information, and monitors user keystrokes and the text returned by the login server. It is one of the most powerful Bro modules for detecting break-ins to Unix systems because of the ability to look for particular commands that attackers often execute once they have penetrated a Unix machine.

The analyzer is generic in the sense that it applies to more than one protocol. Currently, Bro instantiates a login analyzer for both Telnet [RFC-854] and Rlogin [RFC-1282] traffic. In principle, it could do the same for other protocols such as SSH [RFC-FIXME] or perhaps X11 [RFC-1013], if one could write the corresponding elements of the event engine to decrypt the SSH session (naturally, this would require access to the encryption keys) or extract authentication information and keystrokes from the X11 event stream.

Note: The analyzer does an exceedingly limited form of SSH analysis; see hot_ssh_orig_ports.

For Telnet, the event engine knows how to remove in-band Telnet option sequences [RFC-855] from the text stream, and does not deliver these to the event handlers, except for a few options that the engine analyzes in detail (such as attempts to negotiate authentication). Unfortunately, the Telnet protocol does not include any explicit marking of username or password information (unlike the FTP protocol, as discussed in ftp Analyzer). Consequently, Bro employs a series of heuristics that attempt to extract the username and password from the authentication dialog the session is presumed to begin with. The analysis becomes quite complicated due to the possible use of type-ahead and editing sequences by the user, plus the possibility that the user may be an attacker who attempts to mislead the heuristics in order to disguise the username they are accessing.

Analyzing Rlogin is nominally easier than analyzing Telnet because Rlogin has a simpler in-band option scheme, and because the Rlogin protocol explicitly indicates the username in the initial connection dialog. However, this last is not actually a help to the analyzer, because for most Rlogin servers, if the initial username fails authentication (for example, is not present in the .rhosts file local to the server), then the server falls back on the same authentication dialog as with Telnet (prompting for username and then password, or perhaps just for a password to go with the transmitted username). Consequently, the event engine employs the same set of heuristics as for Telnet.

Each connection processed by the analyzer is in a distinct state: user attempting to authenticate, user has successfully authenticated, analyzer is skipping any further processing, or the analyzer is confused (See: login analyzer confusion). You can find out the state of a given connection using get_login_state.

The analyzer uses a capture filter of tcp port 23 or tcp port 513 [[Reference Manual: #Filtering|Filtering</code>]]. It annotates each connection with the username(s) present in the authentication dialog. If the username was authenticated successfully, then it encloses the annotation in quotes. If the authentication failed, then the name is marked as failed/<username>. So, for example, if user "smith" successfully authenticates, then the connection's addl field will have "smith" appended to it:

931803523.006848 254.377 telnet 324 8891 1.2.3.4 5.6.7.8 SF L "smith"

while if "smith" failed to authenticate, the report will look like:

931803523.006848 254.377 telnet 324 8891 1.2.3.4 5.6.7.8 SF L fail/smith

and if they first tried as "smith" and failed, and then succeeded as "jones," the record would look like:

931803523.006848 254.377 telnet 324 8891 1.2.3.4 5.6.7.8 SF L 

fail/smith "jones"

Note: The event engine's heuristics can sometimes get out of synch such that it interprets a password as a username; in addition, users sometimes type their password when they should instead enter their username. Consequently, the connection logs sometimes include passwords in the annotations, and so should be treated as very sensitive information (e.g., not readable by any user other than the one running Bro).

login analyzer confusion

Because there is no well-defined protocol for Telnet authentication (or Rlogin, if the initial .rhosts authentication fails), the login analyzer employs a set of heuristics to detect the username, password, and whether the authentication attempt succeeded. All in all, these heuristics work quite well, but it is possible for them to become confused and reach incorrect conclusions.

Bro attempts to detect such confusion. If it does, then it generates a

event, after which the event engine will no

longer attempt to follow the authentication dialog. In particular, it will not generate subsequent login_failure or login_sucess events. The login_confused event includes a string describing the type of confusion, using one of the values given in the table below.

Different types of confusion that login analyzer can report

Type of confusion	Meaning
"excessive typeahead"	The user has typed ahead 12 or more lines. Deficiency: The upper bound should be adjustable.
"extra repeat text"	The user has entered more than one VMS repeat sequence (an escape followed by "[A") on the same line. Note: Bro determines that a login session involves a VMS server if the server prompts with "Username:". It then interprets VMS repeat sequences as indicating it should replace the current line with the previous line.
"multiple USERs"	The user has specified more than one username using the $USER environment variable.
"multiple login prompts"	The analyzer has seen several login prompts on the same line, and has not seen a corresponding number of lines typed ahead previously by the user.
"no login prompt"	The analyzer has seen 50 lines sent by the server without any of them matching login prompts. Deficiency: The value of 50 should be adjustable.
"no username"	The analyzer is generating an event after having already seen a login failure, but the user's input has not provided another username to include with the event. Note: If the analyzer's heuristics indicate it's okay that no new username has been given, such as when the event is generated due to one connection endpoint closing the connection, then it instead uses the username <none>.
"no username2"	The analyzer saw an additional password prompt without seeing an intervening username, and it has no previous username to reuse.
"non empty multi login"	The analyzer saw multiple adjacent login prompts, with an apparently ignored intervening username typed-ahead between them.
"possible login ploy"	The client sent text that matches one of the patterns reflecting text usually sent by the server. This form of confusion can reflect an attacker attempting to evade the monitor. For example, the client may have sent the text "login: as a username so that when echoed back by the server, the analyzer would misinterpret it as reflecting another login prompt from the server.
"repeat without username"	The user entered a VMS repeat sequence but there is no username to repeat. (See extra repeat text for a discussion of the analyzer's heuristics for dealing with VMS servers.)
"responder environment"	The responder (login server) has signaled a set of environment variables to the originator (login client). This is in the opposite direction as to what makes sense.
"username with embedded repeat"	The line repeated by a VMS server in response to a repeat sequence itself contains a repeat sequence.

login variables

The standard script defines a large number of variables for refining the analysis policy:

• input_trouble : pattern lists patterns that the analyzer should flag if they appear in the user's input (keystroke) stream.

The analyzer searches for these patterns both in the raw text typed by the user and the same lines after applying editing using the edit function twice: once with interpreting BS (ctrl-H) as delete-one-character, and once with DEL as the edit character. If any of these matches, then the analyzer considers the pattern to have matched.

eggdrop Default: a pattern matching occurrences of the strings ``rewt, ``eggdrop, ``loadmodule, or ``/bin/eject. The first of these is a popular username attackers use for root backdoor accounts. The second reflects that one prevalent class of attackers are devotees of Internet Relay Chat (IRC), who frequently upon breaking into an account install the IRC eggdrop utility.

• edited_input_trouble : pattern is the same as input_trouble except the analyzer only checks the edited user input against the pattern, not the raw input (see above).

This variable is provided so you can specify patterns that can occur innocuously as typos; whenever the user corrects the typo before terminating the line, the pattern won't match, because it won't be present in the edited version of the line. In addition, for matches to these patterns, the analyzer delays reporting the match until it sees the next line of output from the server. It then includes both the line that triggered the match and the corresponding response from the server, which makes it easy for a human inspecting the logs to tell if the occurrence of the pattern was in fact innocuous.

Here's an example of an innocuous report:

936723303.760483 1.2.3.4/21550 > 5.6.7.8/telnet
   input "cd ..." yielded output "ksh: ...:  not found."

It was flagged because the user's input included "...", a name commonly used by attackers to surreptitiously hide a directory containing their tools and the like. However, we see from the Telnet server's response that this was not actual access to such a directory, but merely a typing mistake.

On the other hand:

937528764.579039 1.2.3.4/3834 > 5.6.7.8/telnet
   input "cd ..." yielded output "maroon# ftp 

sunspot.sunspot.noao.edu "

shows a problem---the lines returned by the server was a root prompt (``maroon@code{#}), to which the user issued a command to access a remote FTP server.

Deficiency: The analyzer should decouple the notion of waiting to receive the server's reply from the notion of matching only the edited form of the line; there might be raw inputs for which it is useful to see the server's response, and edited inputs for which the server's response is unimportant in terms of knowing that the input spells trouble.

Default: the pattern

   /[ \t]*cd[ \t]+((['"]?\.\.\.)|(["'](\.[^"']*)[ \t]))/

which looks for a "cd" command to either a directory beginning with "..." (optionally quoted by the user) or a directory name beginning with "." that is quoted and includes an embedded blank or tab.

• output_trouble : pattern lists patterns that the analyzer should flag if they occur in the output sent by the login server back to the user.

PATH_UTMP sensitive pattern smashdu.c exploit tool Default: the pattern

     /^-r.s.*root.*\/bin\/(sh|csh|tcsh)/
   | /Jumping to address/
   | /smashdu\.c/
   | /PATH_UTMP/
   | /Log started at =/    
   | /www\.anticode\.com/
   | /smurf\.c by TFreak/
   | /Trojaning in progress/ 
   | /Super Linux Xploit/

The first of these triggers any time the user inspects with the ls utility an executable whose pathname ends in /bin/ followed by one of the popular command shells, and the ls output shows that the command shell has been altered to be setuid to root. The remainder match either the output generated by some popular exploit tools (for example, "Jumping to address", present in many buffer overflow exploit tools), exploit tool names ("smashdu.c"), text found within the tool source code ("smurf.c by TFreak"), or URLs accessed (say via the lynx or fetch utilities) to retrieve attack software ("www.anticode.com").

• backdoor_prompts : pattern lists patterns that the analyzer should flag if they are seen as the first line sent by the server to the user, because they often correspond with backdoors that offer a remote user immediate command shell access without having to first authenticate.

Default: the pattern "/^[!-~]*( ?)[#%$] /", which matches a line that begins with a series of printable, non-blank characters and ends with a likely prompt character, with a blank just after the prompt character and perhaps before it.

• non_backdoor_prompts : pattern lists patterns that if a possible backdoor prompt also matches, then the analyzer should not consider the server output as indicating a backdoor prompt. Used to limit false positives for backdoor_prompts.

Default: the pattern "/^ *#.*#/", which catches lines with more than one occurrence of a #. Some servers generate such lines as part of their welcome banner.

• hot_terminal_types : pattern lists "magic" terminal types sometimes used by attackers to access backdoors. Both Telnet and Rlogin have mechanisms for negotiating a terminal type (name; e.g., "xterm"); these backdoors trigger and skip authentication if the name has a particular value.

VT666 Default: the name "VT666", one of the trigger terminal types we've observed in practice.

• hot_telnet_orig_ports : set[port] Some Telnet backdoors trigger if the ephemeral port used by the client side of the connection happens to be a particular value. This variable is used to list the port values whose use should be considered as possibly indicating a backdoor.

Note: Clearly, this mechanism can generate false positives when the client by chance happens to choose one of the listed ports.

Default: 53982/tcp, one of the trigger ports we have observed in practice.

Deficiency: There should be a corresponding variable for Rlogin backdoors triggered by a similar mechanism.

• hot_ssh_orig_ports : set[port] Similar to hot_telnet_orig_ports, only for SSH.

Default: 31337/tcp, a trigger port that we've observed in practice.

• skip_authentication : set[string] A set of strings that, if present in the server's initial output (i.e., its welcome banner), indicates the analyzer should not attempt to analyze the session for an authentication dialog. This is used for servers that provide public access and don't bother authenticating the user.

Default: the string "WELCOME TO THE BERKELEY PUBLIC LIBRARY", which corresponds to a frequently accessed public server in the Berkeley area. (Obviously, we include this default as an example, and not because it will be appropriate for most Bro users! But it does little harm to include it.)

Deficiency: It would be more natural if this variable and a number of others listed below were of type pattern rather than set[string]. They are actually converted internally by the event engine into regular expressions.

• direct_login_prompts : set[string] A set of strings that if seen during the authentication dialog mean that the user will be logged in as soon as they answer the prompt.

Default: "TERMINAL?", a prompt used by some terminal servers.

login_prompts : set[string] A set of strings corresponding to login username prompts during an authentication dialog.

Default: the strings

   Login:
   login:
   Name:
   Username:
   User:
   Member Name

and the default contents of direct_login_prompts.

• login_failure_msgs : set[string] A set of strings that if seen in text sent by the server during the authentication dialog correspond to a failed login attempt.

Default: the strings

   invalid
   Invalid
   incorrect
   Incorrect
   failure
   Failure,
   User authorization failure,
   Login failed,
   INVALID
   Sorry,
   Sorry.

• login_non_failure_msgs : set[string] A set of strings similar to login_failure_msgs that if present mean that the server text does not actually correspond to an authentication failure (i.e., if login_failure_msgs also matches, it's a false positive).

Default: the strings

   Failures
   failures
   failure since last successful login
   failures since last successful login

• router_prompts : set[string] A set of strings corresponding to prompts returned by the local routers when a user successfully authenticates to the router. For the purpose of this variable, see the next variable.

Default: empty.

• login_success_msgs : set[string] A set of strings that if seen in text sent by the server during the authentication dialog correspond to a successful authentication attempt.

Default: the strings

   Last login
   Last successful login
   Last   successful login
   checking for disk quotas
   unsuccessful login attempts
   failure since last successful login
   failures since last successful login

and the default contents of the router_prompts variable.

Deficiency: Since by default router_prompts is empty, this last inclusion does nothing. In particular, if you redefine router_prompts then login_success_msgs will not pick up the change; you will need to redefine it to (again) include router_prompts, using: redef login_success_msgs += router_prompts. This is clearly a misfeature of Bro and will be fixed one fine day.

• login_timeouts : set[string] A set of strings that if seen in text sent by the server during the authentication dialog correspond to the server having timed out the authentication attempt.

Default: the strings

   timeout
   timed out
   Timeout
   Timed out
   Error reading command input

(This last is returned by the VMS operating system.)

• non_ASCII_hosts : set[addr] A set of addresses corresponding to hosts whose login servers do not (primarily) use 7-bit ASCII. The analyzer will not attempt to analyze authentication dialogs to such hosts, and will not complain about huge lines generated by either the sender or receiver (per excessive_line).

Default: empty.

• skip_logins_to : set[addr] A set of addresses corresponding to hosts for which the analyzer should not attempt to analyze authentication dialogs.

Default: the (empty) contents of non_ASCII_hosts.

• always_hot_login_ids : set[string] A set of usernames that the analyzer should always flag as sensitive, even if they're seen in a session for which the analyzer is confused login analyzer confusion.

Default: the value of always_hot_ids defined by the hot analyzer.

• hot_login_ids : set[string] A set of usernames that the analyzer should flag as sensitive, unless it sees them in a session for which the analyzer is confused (See: login analyzer confusion).

Default: the value of hot_ids defined by the hot-ids analyzer.

• rlogin_id_okay_if_no_password_exposed : set[string] A set of username exceptions to hot_login_ids which the analyzer should not flag as sensitive if the user authenticated without exposing a password (so, for example, via .rhosts).

Default: the username "root".

login functions

The standard login script provides the following functions for external use:

• is_login_conn (c: connection): bool Returns true if the given connection is one analyzed by login (currently, Telnet or Rlogin), false otherwise.

• hot_login (c: connection, msg: string, tag: string) Marks the given connection as hot, logs the given message, and demultiplexes demux the subsequent server-side contents of the connection to a filename based on tag and the client-side to a filename based on the name "keys". No return value.

• is_hot_id (id: string, successful: bool, confused: bool): bool Returns true if the username id should be considered sensitive, given that the user either did or did not successfully authenticate, and that the analyze was or was not in a confused state (See: login analyzer confusion).

• is_forbidden_id (id: string): bool Returns true if the username id is present in forbidden_ids or forbidden_id_patterns.

• edit_and_check_line (c: connection, line: string, successful: bool): check_info Tests whether the given line of text seen on connection c includes a sensitive username, after first applying BS and DEL keystroke editing (see: login variables). successful should be true if the user has successfully authenticated, false otherwise.

The return value is a check_info record, which contains four check_info fields:

• expanded_line All of the different editing interpretations of the line, separated by commas. For example, if the original line is

 ro <DEL><BS><BS> ot

then the different editing interpretations yield the following return value:

 ro <DEL><BS><BS> ot, ro <BS><BS>ot,root

Deficiency: Ideally, these values would be returned in a list of some form, so that they can be accessed separately and unambiguously. The current form is really suitable only for display to a person, and even that can be quite confusing if line happens to contain commas already. Or, perhaps an algorithm of "simply pick the shortest" would find the correct editing every time anyway.

• hot: bool True if any editing sequence resulted in a match against a sensitive username.

• hot_id: string The version of the input line (with or without editing) that was considered hot, or an empty string if none.

• forbidden: bool True if any editing sequence resulted in a match against a username considered ``forbidden, per is_forbidden_id.

• edit_and_check_user (c: connection, user: string, successful: bool, fmt_s: string): bool Tests whether the given username used for authentication on connection c is sensitive, after first applying BS and DEL keystroke editing (See: login variables). successful should be true if the user has successfully authenticated, false otherwise.

fmt_s is a fmt format specifying how the username information should be included in the connection's addl field. It takes two string parameters, the current value of the field and the expanded version of the username as described in expanded_line.

If edit_and_check_line indicates that the username is sensitive, then edit_and_check_user records the connection into its own demultiplexing files . If the username is forbidden, then unless the analyzer is confused, we attempt to terminate the connection using terminate_connection.

Returns true if the connection is now considered ``hot, either due to having a sensitive username, or because it was hot upon entry to the function.

• edit_and_check_password(c: connection, password: string): bool Checks the given password to see whether it contains a sensitive username. If so, then marks the connection as hot and logs the sensitive password. No return value.

Note: The purpose of this function is to catch instances in which the event engine becomes out of synch with the authentication dialog and mistakes what is, in fact, a username being entered, for a password being entered. Such confusion can come about either due to a failure of the event engine's heuristics, or due to deliberate manipulation of the event engine by an attacker.

login event handlers

The standard login script handles the following events:

• login_failure (c: connection, user: string, client_user: string, password: string, line: string) Invoked when the event engine has seen a failed attempt to authenticate as user with password on the given connection c. client_user is the user's username on the client side of the connection. For Telnet connections, this is an empty string, but for Rlogin connections, it is the client name passed in the initial authentication information (to check against .rhosts). line is the line of text that led the analyzer to conclude that the authentication had failed.

The analyzer first generates an account_tried event to facilitate detection of password guessing, and then checks for a sensitive username or password. If the username was not sensitive and the password is empty, then no further analysis is applied, since clearly the attempt was half-hearted and aborted. Otherwise, the analyzer annotates the connection's addl field with fail/@code{<username>} to mark the authentication failure, and also checks the client_user to see if it is sensitive. If we then find that the connection is hot, the analyzer logs a message to that effect.

• login_success (c: connection, user: string, client_user: string, password: string, line: string) Invoked when the event engine has seen a successful attempt to authenticate. The parameters are the same as for login_failure.

The analyzer invokes check_hot with mode APPL_ESTABLISHED since the application session has now been established. It generates an account_tried event to facilitate detection of password guessing, and then checks for a sensitive username or password. The event engine uses the special password "@code{<none>"} to indicate that no password was exposed, and this mitigates the sensitivity of logins using particular usernames per rlogin_id_okay_if_no_password_exposed.

The analyzer annotates the connection's addl field with "@code{<username>"} to mark the successful authentication. Finally, if we then find that the connection is hot, the analyzer logs a message to that effect.

• login_input_line (c: connection, line: string) Invoked for every line of text sent by the client side of the login session to the server side. The analyzer matches the text against input_trouble and edited_input_trouble and invokes hot_login with a tag of "trb" if it sees a match, which will log a notice concerning the connection. However, this invocation is only done while the connection's hot field count is <= 2, to avoid cascaded notices when an attacker gets really busy and steps on a lot of sensitive patterns.

• login_output_line (c: connection, line: string) Invoked for every line of text sent by the server side of the login session to the client side. The analyzer checks backdoor_prompts

and any pending input notices that

were waiting on the server output, per edited_input_trouble. These last are then logged unless the output matched the pattern:

   /No such file or directory/

Deficiency: Clearly, this pattern should not be hardwired but instead specified by a redefinable variable.

Finally, if the line is not too long and the text matches output_trouble and the connection's hot field count is <= 2 (to avoid cascaded notices), the analyzer invokes hot_login with a tag of "trb".

Deficiency: "Too long" is hardwired to be a length > 256 bytes. It, too, should be specifiable via a redefinable variable.

Note: We might wonder if not checking overly long lines presents an evasion threat: the attacker can bury their access to a sensitive string in an excessive line and thus avoid detection. While this is true, it doesn't appear to cost much. First, some of the sensitive patterns are generated in server output that will be hard to manipulate into being overly long. Second, if the attacker is trying to avoid detection, there are easier ways, such as passing their output through a filter that alters it a good deal.

• login_confused (c: connection, msg: string, line: string) Invoked when the event engine's heuristics have concluded that they have become confused and can no longer correctly track the authentication dialog (See: login analyzer confusion). msg gives the particular problem the heuristics detected (for example, multiple_login_prompts means that the engine saw several login prompts in a row, without the type-ahead from the client side presumed necessary to cause them) and line the line of text that caused the heuristics to conclude they were confused.

Once declaring that it's confused, the event engine will no longer attempt to follow the authentication dialog. In particular, it will not generate subsequent login_failure or login_success events.

Upon this event, the standard login script invokes check_hot with mode APPL_ESTABLISHED since it could well be that the application session is now established (it can't know for sure, of course, because the event engine has given up). It annotates the connection's

addl field with

confused<line> to mark the confused state, and then logs to the @file{weird} file the particulars of the connection and the type of confusion (msg).

Deficiency: This should be done by generating a weird-related event instead.

Finally, the analyzer invokes set_record_packets to specify that all of the packets associated with this connection should be recorded to the trace file.

{{note} For the current login analyzer, this call is not needed---it records every packet of every login session anyway, because the generally philosophy is that Bro should record whatever it analyzes, so that the analysis may be repeated or examined in detail. Since the current analyzer looks at every input and output line via login_input and login_output, it records all of the packets of every such analyzed session. There is commented-out text in login_success to be used if login_input and login_output are not being used; it turns off recording of a session's packets after the user has successfully logged in (assuming the connection is not considered hot).

• login_confused_text (c: connection, line: string) Invoked for every line the user types after the event engine has entered the confused state. If the connection is not already considered hot, then the analyzer checks for the presence of sensitive usernames in the line using edit_and_check_line, and, if present, annotates the connection's addl field with confused<line>, logs that the connection has become hot, and invokes set_record_packets to record to the @file{trace} file all of the packets associated with the connection.

• login_terminal (c: connection, terminal: string) Invoked when the client transmits a terminal type to the server. The mechanism by which the client transmits the type depends on the underlying protocol (Rlogin or Telnet).

The handler checks the terminal type against hot_terminal_types and if it finds a match invokes hot_login with a tag of "trb".

• excessive_line (c: connection) Invoked when the event engine observes a very long line sent by either the client or the server. Such long lines are seen as potential attempts by an attacker to evade the login analyzer; or, possibly, as a Login session carrying an unusual application. @emph{Note: One example we have observed occurs when a high-bandwidth binary payload protocol such as Napster is sent over the Telnet or Rlogin well-known port in an attempt to either evade detection or tunnel through a firewall.}

This event is actually generic to any TCP connection carrying an application that uses the ``Network Virtual Terminal (NVT) abstraction, which presently comprises Telnet and FTP. But the only handler defined in the demonstration Bro policy is for Telnet, hence we discuss it here. For this reason, the handler first invokes is_login_conn to check whether the connection is in fact a login session. If so, then if the connection is not hot, and if the analyzer finds the server listed in non_ACSII_HOSTS, then it presumes the long line is due to use of a non-ASCII character set; the analyzer invokes set_login_state and set_record_packets to avoid further analysis or recording of the connection.

Otherwise, if the connection is still in the authentication dialog, then the handler generates a event with a confusion-type of "excessive_line", and changes the connection's state to confused.

Deficiency: The event engine is currently hardwired to consider a line of >= 1024 bytes as "excessive;" clearly this should be user-redefinable.

• inconsistent_option (c: connection) NVT options are specified by the client and server stating which options they are willing to support vs. which they are not, and then instructing one another which in fact they should or should not use for the current connection. If the event engine sees a peer violate either what the other peer has instructed it to do, or what it itself offered in terms of options in the past, then the engine generates an inconsistent_option event.

The handler for this event simply records an entry about it to the

file.  Deficiency: The event handler invocation does not include enough information to determine what option was inconsistently specified; in addition, it would be convenient to integrate the handling of problems like this within the general ``weird framework. 

Note: As for excessive_line above, this event is actually a generic one applicable to any NVT-based protocol. It is handled here because the problem most often crops up for Telnet sessions. Note: Also, the handler does not check to see whether the connection is a login session (as it does for excessive_line); it serves as the handler for any NVT session with an excessive line.

Note: Finally, note that this event can be generated if the session contains a stream of binary data. One way this can occur is when the session is encrypted but Bro fails to recognize this fact.

• bad_option (c: connection) If an NVT option is either ill-formed (e.g., a bad length field) or unrecognized, then the analyzer generates this event.

The processing of this event (recording information to the file) and the various notes and deficiencies associated with it are the same as those for inconsistent_option above.

• bad_option_termination (c: connection) If an NVT option fails to be terminated correctly (for example, a character is seen within the option that is disallowed for use in the option), then the analyzer generates this event.

The processing of this event (recording information to the file) and the various notes and deficiencies associated with it are the same as those for inconsistent_option above.

• authentication_accepted (name: string, c: connection) The NVT framework includes options for negotiating authentication. When such an option is sent from client to server and the server replies that it accepts the authentication, then the event engine generates this event.

The handler annotates the connection's addl field with auth<name>, unless that annotation is already present.

• authentication_rejected (name: string, c: connection) The same as authentication_accepted, except invoked when the server replies that it rejects the attempted authentication.

The handler annotates the connection's addl field with auth-failed<name>.

• authentication_skipped (c: connection) Invoked when the event engine sees a line in the authentication dialog that matches .

The handler annotates the connection's addl field with "skipped" to mark that authentication was skipped, and then invokes skip_further_processing and (unless the connection is hot) set_record_packets to skip any further analysis of the connection, and to stop recording its packets to the trace file.

• connection_established (c: connection) connection_established is a generic event generated for all TCP connections; however, the login analyzer defines an additional handler for it.

The handler first checks (via is_login_conn) whether this is a Telnet or Rlogin connection. If so, it generates an authentication_skipped

event if the server's address occurs

in skip_logins_to, and also (for Telnet) checks whether the client's port occurs in hot_telnet_orig_ports, invoking hot_login

with the tag "orig" if it does.

For SSH connections, it likewise checks the client's port, but in hot_ssh_orig_ports, marking the connection as hot and logging a real-time notice if it is.

• partial_connection (c: connection) As noted earlier, partial_connection is a generic event generated for all TCP connections. The login analyzer also defines a handler for it, one which (if it's a Telnet/Rlogin connection) sets the connection's state to confused and checks for hot_telnet_orig_ports.

• activating_encryption (c: connection) The NVT framework includes options for negotiating encryption. When such a series of options is successfully negotiated, the event engine generates this event.

Note: The negotiation sequence is complex and can fail at a number of points. The event engine does not attempt to generate events for each possible failure, but instead only looks for the option sent after a successful negotiation sequence.

The handler annotates the connection's addl field with ``(encrypted) to mark that authentication was encrypted.

Note: The event engine itself marks the connection as requiring no further processing. This is done by the event engine rather than the handler because the event engine cannot do its job (regardless of the policy the handler might desire) in the face of encryption.

The pop3 Analyzer

The pop3 analyzer does a protocol analysis of the Post Office Protocol - Version 3.

When Bro runs with the pop3 Analyzer, it processes all packets with destination port 110/tcp, generating a log file pop3.log. Each line contains a timestamp, a connection ID, the originator and responder IP addresses, and the message sent. The message consists of the command and arguments on client side, and the status on server side.

The pop3_session_info record

The pop3 analyzer maintains a pop3_session_info record per pop3 connection:

type pop3_session_info: record {
   id: count;              # Unique session ID.
   quit_sent: bool;        # Client issued a QUIT.
   last_command: string;   # Last command of client.
};

The corresponding fields are:

• id The unique session identifier assigned to this session. Sessions are numbered starting at 1 and incremented with each new session.

• quit_sent True if the client has sent a QUIT command.

• last_command Last command issued by the client.

pop3 variables

• pop_connections: table[conn_id] of pop3_session_info This table contains all active POP3-sessions indexed by their Connection IDs. As soon as the TCP Connection terminates or expires, they are deleted. * pop_connection_weirds: table[addr] of count &default=0 &create_expire = 5 mins This table contains all the POP3-session originators for which unexpected behavior was recorded. * error_threshold: count = 3 This variable contains a threshold for the maximum number of negative status indicators per originator received from a server. It is used for recognizing potential abuses, e.g., trial and error password guessing attacks. * ignore_commands: set[string] Set of commands to ignore while generating the log file.

pop3 event handlers

• pop3_request(c: connection, is_orig: bool, command: string, arg: string) Generated for each valid command sent from the client to the server.
• pop3_reply(c: connection, is_orig: bool, cmd: string, msg: string) Generated for each server reply containing a valid status indicator.
• pop3_data(c: connection, is_orig: bool, data: string) Generated for every data line sent by the server as a reply to the client, including commands that yield multi-line answers.
• pop3_unexpected(c: connection, is_orig: bool, msg: string, detail: string) Generated when something semantically unexpected has happened.
• pop3_login_success(c: connection, is_orig: bool, user: string, password: string) Generated when a user authenticates successfully. The password may be empty if it has not been observed.
• pop3_login_failure(c: connection, is_orig: bool, user: string, password: string) Generated when a user fails to authenticate correctly.

The portmapper Analyzer

The portmapper analyzer monitors one particularly important form of remote procedure call (RPC) [RFC-1831, RFC-1832] traffic: the portmapper service, used to map between RPC program (and version) numbers and the TCP or UDP port on which the service runs for a particular host. For example, rstatd is an RPC service that provides "remote host status monitoring" so that a set of hosts can be informed when any of them reboots. rstatd has been assigned a standard RPC program number of 100002. To find out the corresponding TCP or UDP port on a given host, a remote host would usually first contact the portmapper RPC service running on the host and request the port corresponding to program 100002.

Types of calls to the RPC portmapper service

Call	Meaning
NULL	A do-nothing call typically provided by all RPC services.
GETPORT	Look up the port associated with a given RPC program.
SET	Add a new port mapping (or replace an existing mapping) for an RPC program.
UNSET	Remove a port mapping.
DUMP	Retrieve all of the RPC program mappings.
CALLIT	Both look up a program and then directly call it.

All in all, clients can make six different types of calls to the portmapper, as summarized in the above table. Attackers often use GETPORT and DUMP to see whether a host may be running an RPC service vulnerable to a known exploit.

The analyzer uses a capture filter of ``port 111 (See: Filtering), equivalent to ``tcp port 111 or udp port 111 (since the portmapper service ordinarily accepts calls using either TCP or UDP, both on port 111). It checks the different types of portmapper calls against policies expressed using a number of different variables.

Note: An important point not to overlook is that an attacker does not have to first call the portmapper service in order to call an RPC program. They might instead happen to know the port on which the service runs a priori, since for example it may generally run on the same port for a particular operating system; or they might scan the host's different TCP or UDP ports directly looking for a reply from the service. Thus, while portmapper monitoring proves very useful in detecting attacks, it does not provide comprehensive monitoring of attempts to exploit RPC services.

portmapper variables

The standard script provides the following redefinable variables:

• rpc_programs : table[count] of string Maps RPC program numbers to a string used to name the service. For example, the [100002] entry is mapped to "rstatd".

Default: a large list of RPC services.

• NFS_services : set of string Lists the names of those RPC services that correspond to Network File System (NFS) [RFC-1094, RFC-1813] services. This variable is provided because it is convenient to express policies specific to accessing NFS file systems.

Default: the services mountd, nfs, pcnfsd, nlockmgr, rquotad, status.

Deficiency: Bro's notion of NFS is currently confined to just knowledge of the existence of these services. It does not analyze the particulars of different NFS operations.

• RPC_okay : set[addr, addr, string] Indexed by the host providing a given service and then by the host accessing the service. If an entry is present, it means that the given access is allowed.

For example, an entry of:

   [1.2.3.4, 5.6.7.8, "rstatd"]

means that host 5.6.7.8 is allowed to access the rstatd service on host 1.2.3.4.

Default: empty.

• RPC_okay_nets : set[net] A set of networks allowed to make GETPORT requests without complaint. The notion behind providing this variable is that the listed networks are trusted. However, the trust doesn't extend beyond GETPORT to other portmapper requests, because GETPORT is the only portmapper operation used routinely by a set of hosts trusted by another set of hosts (but that don't belong to the same group, and hence are not issuing SET and UNSET calls).

Default: empty.

• RPC_okay_services : set[string] A set of services for which GETPORT requests should not generate complaints. These might be services that are widely invoked and believed exploit-free, such as walld, though care should be taken with blithely assuming that a given service is indeed exploit-free. Note that, like for RPC_okay_nets, the trust does not extend beyond GETPORT, because it should be the only portmapper operation routinely invoked.

Default: empty.

• NFS_world_servers : set[addr] A set of hosts that provide public access to an NFS file system, and thus should not have any of their NFS traffic flagged as possibly sensitive. (The presumption here is that such public servers have been carefully secured against any remote NFS operations.) An example of such a server might be one providing read-only access to a public database.

Default: empty.

• RPC_dump_okay : set[addr, addr] Indexed first by the host requesting a portmapper dump, and second by the host from which it's requesting the dump. If an entry is present, then the dump operation is not flagged.

Default: empty.

• any_RPC_okay : set[addr, string] Pairs of hosts and services for which any GETPORT access to the given service is allowed.

• sun-rpc.mcast.net

Default:

   [NFS_world_servers, NFS_services],
   [sun-rpc.mcast.net, "ypserv"]

The first of these allows access to any NFS service of any of the NFS_world_servers, using Bro's cross-product initialization feature (See Initializing Tables). The second allows "ypserv" requests to the multicast address reserved for RPC multicasts. (I don't know how much this type of access is actually used in practice, but experience shows that requests for "ypserv" directed to that address pop up not infrequently.)

• suppress_pm_log : table[addr, string] of bool Do not generate real-time notices for access by the given address for the given service. Note that unlike most Bro policy variables, this one is not const but is modified at run-time to add to it any host that invokes the "walld" RPC service, so that such access is only reported once for each host.

Default: empty, but dynamic as discussed above.

portmapper functions

The standard script provides the following externally accessible functions:

• rpc_prog (p: count): string Returns the name of the RPC program with the given number, if it's present in ; otherwise returns the text "unknown-@code{<p>"}.

• pm_check_getport (r: connection, prog: string): bool Checks a GETPORT request for the given program against the policy expressed by RPC_okay_services, any_RPC_okay, RPC_okay, and RPC_okay_nets, returning true if the request violates policy, false if it's allowed.

• pm_activity (r: connection, log_it: bool) A bookkeeping function invoked when there's been portmapper activity on the given connection.

The function records the connection via , unless it is a TCP connection (which will instead be recorded by connection_finished). If log_it is true then the function generates a real-time notice of the form:

 rpc:
   <connection-id >
   <RPC-service >
   <r$addl >

For example:

 972616255.679799 rpc: 65.174.102.21/832 182.7.9.47/portmapper pm_getport: nfs -> 2049/udp

However, it does not generate the notice if either the client host and service are present in suppress_pm_log, or if it already generated a notice in the past for the same client, server and service (to prevent notice cascades).

• pm_request (r: connection, proc: string, addl: string, log_it: bool) Invoked when the given connection has made a portmapper request of some sort for the given RPC procedure proc. addl gives an annotation to add to the connection's addl field. If log_it is true, then connection should be logged; it will also be logged if the function determines that it is hot.

The function first invokes check_scan and scan_hot (with a mode of CONN_ESTABLISHED), unless r is a TCP connection, in which case these checks have already been made by connection_established. The function then adds addl to the connection's addl field, though if the field's length already exceeds 80 bytes, then it just tacks on "..." (unless already present). This last is necessary because Bro will sometimes see zillions of successive portmapper requests that all use the same connection ID, and these will each add to addl until it becomes unwieldy in size. Deficiency: Clearly, the byte limit of 80 should be adjustable.

Finally, the function invokes check_hot with a mode of CONN_FINISHED, and pm_activity to finish up bookkeeping for the connection.

No return value.

• pm_attempt (r: connection, proc: string, status: count, addl: string, log_it: bool) Invoked when the given connection attempted to make a portmapper request of some sort, but the request failed or went unanswered. The arguments are the same as for pm_request, with the addition of status, which gives the RPC status code corresponding to why the attempt failed (see below).

The function first invokes check_scan and check_hot (with a mode of CONN_ATTEMPTED), unless r is a TCP connection, in which case these checks have already been made by connection_attempt.

The function then adds addl to the connection's addl field, along with a text description of the RPC status code, as given in the Table below.

No return value.

Types of RPC status codes

Status description	Meaning
"ok"	The call succeeded.
"prog unavail"	The call was for an RPC program that has not registered with the portmapper.
"mismatch"	The call was for a version of the RPC program that has not registered with the portmapper.
"garbage args"	The parameters in the call did not decode correctly.
"system err"	A system error (such as out-of-memory) occurred when processing the call.
"timeout"	No reply was received within 24 seconds of the request.
"auth error"	The caller failed to authenticate to the server, or was not authorized to make the call.
"unknown"	An unknown error occurred.

portmapper event handlers

The standard script handles the following events:

• pm_request_null (r: connection) Invoked upon a successful portmapper request for the ``null procedure. The script invokes pm_request with log_it=F.

• pm_request_set (r: connection, m: pm_mapping, success: bool) Invoked upon a nominally successful portmapper request to set the portmapper binding m. The script invokes pm_request with log_it=T. success is true if the server honored the request, false otherwise; the script turns this into an annotation of "ok" or "failed".

The pm_mapping type (for m) has three fields, program: count, version: count and p: port, the port for the mapping of the given program and version. pm_mapping

• pm_request_unset (r: connection, m: pm_mapping, success: bool) Invoked upon a nominally successful portmapper request to remove a portmapper binding. The script invokes pm_request with log_it=T. success is true if the server honored the request, false otherwise; the script turns this into an annotation of "ok" or "failed".

• pm_request_getport (r: connection, pr: pm_port_request, p: port) Invoked upon a successful portmapper request to look up a portmapper binding. pr, of type pm_port_request, has three fields: program: count, version: count, and is_tcp: bool, this last indicating whether the caller is request the TCP or UDP port, if the given program/version has mappings for both. The script invokes pm_request with log_it set according to the return value of and an annotation of the mapping.

• pm_request_dump (r: connection, m: pm_mappings) Invoked upon a successful portmapper request to dump the portmapper bindings. The script invokes pm_request with log_it=T unless indicates that the dump call is allowed. The script ignores m, which gives the mappings as a table[count] of pm_mapping, where the table index simply reflects the order in which the mappings were returned, starting with an index of 1. Deficiency: What the script should do, instead, is keep track of the mappings so that Bro can identify the service associated with connections for otherwise unknown ports. }

• pm_request_callit (r: connection, pm_callit_request, p: port) Invoked upon a successful portmapper request to look up and call an RPC procedure. The script invokes pm_request with log_it=T unless the combination of the caller and the program are in suppress_pm_log. Finally, if the program called is walld, then the script adds the caller to suppress_pm_log.

The pm_callit_request type has four fields: pm_callit_request program: count, version: count, proc: count, and arg_size: count. These reflect the procedure being looked up and called, and the size of the arguments being passed to it, respectively. Deficiency: Currently, the event engine does not do any analysis or refinement of the arguments passed to the procedure (such as making them available to the event handler) or the return value. p is the port value returned by the call.

• pm_attempt_null (r: connection, status: count) Invoked upon a failed portmapper request for the ``null procedure. status gives the reason for the failure. The script invokes pm_attempt with log_it=T.

• pm_attempt_set (r: connection, status: count, m: pm_mapping) Invoked upon a failed portmapper request to set the portmapper binding m. The script invokes pm_attempt with log_it=T.

• pm_attempt_unset (r: connection, status: count, m: pm_mapping) Invoked upon a failed portmapper request to remove a portmapper binding. The script invokes pm_attempt with log_it=T.

• pm_attempt_getport (r: connection, status: count, pr: pm_port_request) Invoked upon a failed portmapper request to look up a portmapper binding. pr, of type pm_port_request, has three fields: program: count, version: count, and is_tcp: bool, this last indicating whether the caller requested the TCP or UDP port. The script invokes pm_attempt with log_it set according to the return value of pm_check_get_port.

• pm_attempt_dump (r: connection, status: count) Invoked upon a failed portmapper request to dump the portmapper bindings. The script invokes pm_attempt with log_it=T unless RPC_dump_okay indicates that the dump call is allowed.

• pm_attempt_callit (r: connection, status: count, pm_callit_request) Invoked upon a failed portmapper request to look up and call an RPC procedure. The script invokes pm_attempt with log_it=T unless the combination of the caller and the program are in suppress_pm_log. Finally, if the program called is walld, then the script adds the caller to suppress_pm_log.

• pm_bad_port (r: connection, bad_p: count) Invoked when a portmapper request or response includes an invalid port number. Since ports are represented by unsigned 4-byte integers, they can stray outside the allowed range of 0--65535 by being >= 65536. The script invokes conn_weird_log with a weird tag of "bad_pm_port".

The analy Analyzer

The analy analyzer provides a limited mechanism to use Bro to do statistical analysis on TCP connections. Its primary purpose is to demonstrate that Bro has applications to network traffic analysis beyond intrusion detection. It defines one event handler:

• conn_stats c: connection, os: endpoint_stats, rs: endpoint_stats Invoked for each connection when it terminates (for whatever reason). os and rs are the statistics for the originator endpoint and the responder endpoint, respectively; the table below gives the different record fields.

endpoint_stats fields for summarizing connection endpoint statistics, all of type count.

endpoint_stats fields for summarizing connection endpoint statistics, all of type count

Field	Meaning
num_pkts	The number of packets sent by the endpoint, as seen by the monitor. The endpoint may have sent others that the network dropped upstream from the monitor.
num_rxmit	The number of packets retransmitted by the endpoint, as seen by the monitor.
num_rxmit_bytes	The number of bytes retransmitted by the endpoint.
num_in_order	The number of packets sent by the endpoint that arrived at the monitor in order, where "in order" means in the same order as sent by the endpoint, rather than in sequence number. (Thus, a retransmission can arrive in order, by this definition.) Bro determines if the packet arrived in order by applying heuristics to the IP identification (ID) field, which in general will increase by a small amount between successive packets transmitted by an endpoint.
num_OO	The number of packets sent by the endpoint that arrived at the monitor out of order. See the previous entry for the definition of "in order", and hence "out of order".
num_repl	The number of extra copies of packets sent by the endpoint that arrived at the monitor. Bro considers a packet replicated if its IP ID field is the same as for the previous packet it saw from the endpoint. Using this definition, a replication is most likely caused by a network mechanism such as duplication of a packet by a router, rather than a transport mechanism such as retransmission, though some TCPs fully reuse packets when retransmitting them, including their IP ID field.
endian_type	Whether the advance of the IP ID field as seen by the monitor was consistent with bigendian (network order) addition, little-endian, or undetermined. The three values are represented by the Bro constants ENDIAN_BIG, ENDIAN_LITTLE, and ENDIAN_UNKNOWN. In addition, the value can be ENDIAN_CONFUSED, meaning that the monitor saw conflicting evidence for little- and big-endian.

The signature Analysis Script

The signature module analyzes signature matches (see Signatures). For each signature, you can specify one of the actions defined in Table 7.2. In addition, the module identifies two types of exploit scans: horizontal (a host triggers a signature for multiple destinations) and vertical (a host triggers multiple signature for the same destination).

The module handles one event:

• signature_match (state: signature_state, msg: string, data: string) Invoked upon a match of a signature which contains an event action (See Actions).

It provides the following redefinable variables:

• sig_actions : table[string] of count Maps signature IDs to actions as defined in the table below.

Possible actions to take for signatures matches

Action	Meaning
SIG_IGNORE	Ignore the signature completely.
SIG_QUIET	Process for scan detection but don't report individually.
SIG_FILE	Write matches to signatures-log
SIG_FILE_BUT_NOT_SCAN	Same, but ignore for scan processing
SIG_ALARM	Alarm and write to signatures, notice, and alarm files
SIG_ALARM_ONCE	Same, but only for the first instance
SIG_ALARM_PER_ORIG	Same, but once per originator
SIG_ALARM_NO_WORM	Same, but ignore if generated by known worm-source
SIG_COUNT_PER_RESP	Count per destination and alarm if threshold reached
SIG_SUMMARY	Don't alarm, but generate per-originator summary

Default: SIG_FILE.

• horiz_scan_thresholds : set[count] Generate a notice whenever a remote host triggers a signature for the given number of hosts.

Default: { 5, 10, 50, 100, 500, 1000 }

• vert_scan_thresholds : set[count] Generate a notice whenever a remote host triggers the given number of signatures for the same destination.

Default: { 5, 10, 50, 100, 500, 1000 }

The module defines one function for external use:

• has_signature_matched (id: string, orig: addr, resp: addr): bool Returns true if the given signature has already matched for the (originator,responder) pair.

The SSL Analyzer

The SSL analyzer processes traffic associated with the SSL (Secure Socket Layer) protocol versions 2.0, 3.0 and 3.1. SSL version 3.1 is also known as TLS (Transport Layer Security) version 1.0 since from that version onward the IETF has taken responsibility for further development of SSL.

Bro instantiates an SSL analyzer for any connection with service ports 443/tcp (https), 563/tcp (nntps), 585/tcp (imap4-ssl), 614/tcp (sshell), 636/tcp (ldaps), 989/tcp (ftps-data), 990/tcp (ftps), 992/tcp (telnets), 993/tcp (imaps), 994/tcp (ircs), 995/tcp (pop3s), providing you have loaded the SSL analyzer, or defined a handler for one of the SSL events.

By default, the analyzer uses the above set of ports as a capture filter (See: Filtering). It currently checks the SSL handshake process for consistency, tries to verify seen certificates, generates several events, does connection logging, tries to detect security weaknesses, and produces simple statistics. It is also able to store seen certificates on disk. However, it does no decryption, so analysis is limited to clear text SSL records. This means that analysis stops in the middle of the handshaking phase for SSLv2 and at the end of it for SSLv3.0/SSLv3.1 (TLS). For this reason we have not implemented the SSL session caching mechanism (yet).

The analyzer consists of the four files: ssl.bro, ssl-ciphers.bro, ssl-errors.bro, and ssl-alerts.bro, which are accessed by @load ssl. The analyzer writes to the weird and ssl log files. The first receives all non-conformant and ``weird activity, while the latter tracks the SSL handshaking phase.

The x509 record

This record is a very simplified structure for storing X.509 certificate information. It currently supports only the issuer and subject names.

type x509: record {
   issuer:  string; # issuer name of the certificate
   subject: string; # subject name of the certificate
};

The ssl_connection_info record

The main data structure managed by the SSL analyzer is a collection of ssl_connection_info records, where the record type is shown below.

type ssl_connection_info: record {
id: count;                      # the log identifier number
connection_id: conn_id;         # IP connection information
version: count;                 # version associated with connection
client_cert: x509;
server_cert: x509;
id_index: string;               # index for associated sessionID
handshake_cipher: count;        # cipher suite client and server agreed upon
};

The corresponding fields are {{fixme}FIXME: the description here is out of date}:

• id The unique connection identifier assigned to this connection. Connections are numbered starting at 1 and incrementing with each new connection.

• connection_id The TCP connection which this SSL connection is based on.

• version The SSL version number for this connection. Possible values are SSLv20, for SSL version 2.0, SSLv30 for version 3.0, and SSLv31 for version 3.1.

• client_cert The information from the client certificate, if available.

• server_cert The information from the server certificate, if available.

• id_index Index into associated SSL_sessionID_record table.

• handshake_cipher The cipher suite client and server agreed upon. Note: For SSLv2 cached sessions, this is a placeholder (0xABCD).

SSL variables

The standard script defines the following redefinable variables:

• ssl_compare_cipherspecs : bool If true, remember the client and server cipher specs and perform additional tests. This costs an extra amount of memory (normally only for a short time) but enables detection of non-intersecting cipher sets, for example.

Default: T.

• ssl_analyze_certificates : bool If true, analyze certificates seen in SSL connections, which includes the following steps: * Generating a hash of the certificate and checking if we alreadysaw it earlier from the current host. If so, we won'tverify it, because we already did and verifying is acomputational expensive process. If the certificate haschanged for the current host, generate a weird event. * Verify the certificate. * Store of the certificate on disk in DER format. Default: T.

• ssl_store_certificates : bool If certificates are analyzed, this variable determines they should be stored on disk.

Default: T.

• ssl_store_cert_path : string Path where certificates are stored. If empty, use the current directory. Note: The path must not end with a slash!

Default: "../certs".

• ssl_verify_certificates : bool If certificates are analyzed, whether to verify them.

Default: T.

• x509_trusted_cert_path : string Path where OpenSSL looks for trusted certificates. If empty, use the default OpenSSL path.

Default: "".

• ssl_max_cipherspec_size : count Maximum size in bytes for an SSL cipherspec. If we see attempted use of larger cipherspecs, warn and skip comparing it.

Default: 45.

• ssl_store_key_material : bool If true, stores key material exchanged in the handshaking phase. Note: This is mainly for decryption purposes and currently useless.

Default: T.

@float Figure, SSL example

1046778101.534846 #1 192.168.0.98/32988 > 

213.61.126.124/https start

1046778101.534846 #1 connection attempt version: 3.1
1046778101.534846 #1 cipher suites: SSLv3x_RSA_WITH_RC4_128_MD5 (0x4), 

SSLv3x_RSA_FIPS_WITH_3DES_EDE_CBC_SHA (0xFEFF), SSLv3x_RSA_WITH_3DES_EDE_CBC_SHA (0xA), SSLv3x_RSA_FIPS_WITH_DES_CBC_SHA (0xFEFE), SSLv3x_RSA_WITH_DES_CBC_SHA(0x9), SSLv3x_RSA_EXPORT1024_WITH_RC4_56_SHA (0x64), SSLv3x_RSA_EXPORT1024_WITH_DES_CBC_SHA (0x62), SSLv3x_RSA_EXPORT_WITH_RC4_40_MD5 (0x3), SSLv3x_RSA_EXPORT_WITH_RC2_CBC_40_MD5 (0x6),

1046778101.753356 #1 server reply, version: 3.1
1046778101.753356 #1 cipher suite: SSLv3x_RSA_WITH_RC4_128_MD5 (0x4),
1046778101.762601 #1 X.509 server issuer: /C=DE/ST=Hamburg/L=Hamburg/O=TC 

TrustCenter for Security in Data Networks GmbH/OU=TC TrustCenter Class 3 CA/Email=certificate@trustcenter.de,

1046778101.762601 #1 X.509 server subject: /C=DE/ST=Berlin/O=Lehmanns 

Fachbuchhandlung GmbH/OU=Zentrale EDV/CN=www.jfl.de/Email=admin@lehmanns.de

1046778101.894567 #1 handshake finished, version 3.1, cipher suite: 

SSLv3x_RSA_WITH_RC4_128_MD5 (0x4)

1046778104.877207 #1 finish
---

Used cipher-suites statistics:
SSLv3x_RSA_WITH_RC4_128_MD5 (0x4): 1

@caption{Example of SSL log file with a single SSL session.} @end float

In addition, ssl_log holds the name of the SSL log file to which Bro writes SSL connection summaries. It defaults to open_log_file("ssl").

The above figure shows an example of how entries in the SSL log file look like. We see a transcript of the first SSL connection seen since Bro started running. The first line gives its start and the participating hosts and ports. Next, we see a client trying to attempt a SSL (Version 3.1) connection and the cipher suites offered. The server replies with a SSL 3.1 SERVER-REPLY and the desired cipher suite. Note: In SSL v3.0/v3.1 this determines which cipher suite will be used for the connection. Following this is the certificate the server sends, including the issuer and subject. Finally, we see that the handshaking phase for this SSL connection is finished now, and that client and server agreed on the cipher suite: RSA_WITH_RC4_128_MD5. Due to encryption, the SSL analyzer skips all further data. We only see the end of the connection. When Bro finishes, we get some statistics about the cipher suites used in all monitored SSL connections.

SSL event handlers

The standard script handles the following events:

• ssl_conn_attempt (c: connection, version: count, cipherSuites: cipher_suites_list) Invoked upon the client side of connection c when the analyzer sees a CLIENT-HELLO of SSL version version including the cipher suites the client offers cipherSuites.

The version can be 0x0002, 0x0300 or 0x0301. A new entry is generated inside the SSL connection table and the cipher suites are listed. Ciphers, that are known as weak (according to a corresponding table of weak ciphers) are logged inside the weak.log file. This also happens to cipher suites that we do not know yet. Note: See the file ssl-ciphers.bro for a list of known cipher suites.

• ssl_conn_server_reply (c: connection, version: count, cipherSuites: cipher_suites_list) This event is invoked upon the analyzer receiving a SERVER-HELLO of the SSL server. It contains the SSL version the server wishes to use (Note: This finally determines, which SSL version will be used further) and the cipher suite he offers. If it is SSL version 3.0 or 3.1, the server determines within this SERVER-HELLO the cipher suite for the following connection (so it will only be one). But if it's a SSL version 2.0 connection, the server only announces the cipher suites he supports and it's up to the client to decide which one to use.

Again, the cipher suites are listed and weak and unknown cipher suites are reported inside weak.log.

• ssl_certificate_seen (c: connection, isServer: int) Invoked whenever we see a certificate from client or server but before verification of the certificate takes place. This may be useful, if you want to do something before certificate verification (e.g. do not verify certificates of some given servers).

• ssl_certificate (c: connection, cert: x509, isServer: bool) Invoked after the certificate from server or client (isServer) has been verified. Note: We only verify certificates once. If we see them again, we only check if they have changed! cert holds the issuer and subject of the certificate, which gets stored inside this SSL connection's information record inside the SSL connection table and are written to ssl.log.

• ssl_conn_reused (c: connection, session_id: string) Invoked whenever a former SSL session is reused. session_id holds the session ID as string of the reused session and is written to ssl.log. Currently we don't do session tracking, because SSL version 2.0 doesn't send the session ID in clear text when it's generated.

• ssl_conn_established (c: connection, version: count, cipher_suite: count) Invoked when the handshaking phase of an SSL connection is finished. We see the used SSL version and the cipher suite that will be used for cryptography (written to ssl.log) if we have SSL version 3.0 or 3.1. In case of SSL version 2.0 we can only determine the used cipher suite for new sessions, not for reused ones. (@emph{Note: In SSL version 3.0 and 3.1 the cipher suite to be used is already announced in the SERVER-HELLO.})

• ssl_conn_alert (c: connection, version: count, level: count, description: count) Invoked when the analyzer receives an SSL alert. The level of the alert (warning or fatal) and the description are written into ssl.log. (Note: See ssl-alerts.bro).

• ssl_conn_weak (name: string, c: connection) This event is called when the analyzer sees: * weak ciphers (See: ssl_conn_attempt, ssl_server_reply, ssl_conn_established), * unknown ciphers (See: ssl_conn_attempt, ssl_server_reply, ssl_conn_established) * or certificate verification failed.

See weak.bro.

The weird Analysis Script

The weird module processes unusual or exceptional events. A number of these ``shouldn't or even ``can't happen, yet they do. The general design philosophy of Bro is to check for such events whenever possible, because they can reflect incorrect assumptions (either Bro's or the user's), attempts by attackers to confuse the monitor and evade detection, broken hardware, misconfigured networks, and so on.

Weird events are divided into three categories, namely those pertaining to: connections; flows (a pair of hosts, but for which a specific connection cannot be identified); and network behavior (cannot be associated with a pair of hosts). These categories have a total of four event handlers: conn_weird, conn_weird_addl, flow_weird, and net_weird, and in the corresponding sections below we catalog the events handled by each. In addition, we separately catalog the events generated by the standard scripts themselves (See: Events generated by the standard scripts). Finally, two more weird events have their own handlers, in order to associate detailed information with the event: rexmit_inconsistency and ack_above_hole.

weird_file is the logging file that the module uses to record exceptional events. It defaults to open_log_file("weird").

Note: While these events shouldn't happen, in reality they often do. For example, of the 73 listed below, a search of 10 months' worth of logs at LBNL shows that 42 were seen operationally. While some of the instances reflect attacks, the great majority are simply due to i) buggy implementations, ii) diverse use of the network, or iii) Bro bugs or limitations. Accordingly, you may initially be inclined to log each instance, but don't be surprised to find that you soon decide to only record many of them in the weird file, or not record them at all. (For further discussion, see the section on "crud" in [Pa99].)

Actions for "weird" events

The general approach taken by the module is to categorize for each event the action to take when the event engine generates the event. Table XX summarizes the different possible actions.

Different types of possible actions to take for "weird" events

Action	Meaning
WEIRD_UNSPECIFIED	No action specified.
WEIRD_IGNORE	Ignore the event.
WEIRD_FILE	Record the event to weird file, if it has not been seen for these hosts before. (But see weird do not ignore repeats.)
WEIRD_NOTICE_ALWAYS	Record the event to weird file and generate a notice each time the event occurs.
WEIRD_NOTICE_ONCE	Record the event to weird file; generate a notice the first time the event occurs.
WEIRD_NOTICE_PER_CONN	Record the event to weird file; generate a notice the first time it occurs for a given connection.
WEIRD_NOTICE_PER_ORIG	Record the event to weird file; generate a notice the first time it occurs for a given originating host.

weird variables

The standard weird script provides the following redefinable variables:

• weird_action : table[string] of count Maps different weird events to actions as given in Table in Actions for weird events above.

Default: as specified in conn_weird, conn_weird_addl, flow_weird, net_weird, and Events generated by the standard scripts. As usual, you can change particular values using refinement. For example:

redef weird_action: table[string] of count += {
   [ ["bad_TCP_checksum", "bad_UDP_checksum"] ] = WEIRD_IGNORE,
   ["fragment_overlap"] = WEIRD_NOTICE_PER_CONN,
};

would specify to ignore TCP and UDP checksum errors (rather than the default of WEIRD_FILE), and to notice fragment overlaps once per connection in which they occur, rather than the default of WEIRD_NOTICE_ALWAYS.

• weird_action_filters : table[string] of function(c: connection): count Indexed by the name of a weird event, yields a function that when called for a given connection exhibiting the event, returns an action from the table in section Actions for weird events. A return value of WEIRD_UNSPECIFIED means ``no special action, use the action you normally would. This variable thus allows arbitrary customization of the handling of particular events.

Default: empty, for the weird analyzer itself. The analyzer redefines this variable as follows:

redef weird_action_filters += {
       [ ["bad_RPC", "excess_RPC", "multiple_RPCs", 
	  "partial_RPC"] ] = RPC_weird_action_filter,
};

where RPC_weird_action_filter is a function internal to the analyzer that returns WEIRD_FILE if the originating host is in , and WEIRD_UNSPECIFIED otherwise.

• weird_ignore_host : set[addr, string] Specifies that the analyzer should ignore the given weird event (named by the second index) if it involves the given address (as either originator or responder host).

Default: empty.

• weird_do_not_ignore_repeats : set[string] Gives a set of weird events that, if their action is WEIRD_FILE, should still be recorded to the weird_file each time they occur.

Default: the events relating to checksum errors, i.e., "bad_IP_checksum", "bad_TCP_checksum", "bad_UDP_checksum", and "bad_ICMP_checksum". These are recorded multiple times because it can prove handy to be able to track clusters of checksum errors.

weird functions

The weird analyzer includes the following functions:

• report_weird (t: time, name: string, id: string, action: WeirdAction, no_log: bool) Processes an occurrence of the weird event name associated with the connection described by the string id (which may be empty if no connection is associated with the event). action is the action associated with the event. For report_weird, the only distinctions made between the different actions are that WEIRD_IGNORE causes the function to do nothing; any of WEIRD_NOTICE_xxx cause the function to generate a notice, unless no_log is true; and WEIRD_UNSPECIFIED causes the function to look up the action in weird_action. If the function does not find an action for the event, then it uses WEIRD_NOTICE_ALWAYS and prepends the log message with a pair of asterisks (``**) to flag that this event does not have a specified action.

For WEIRD_FILE, report_weird only records the event once to the file, unless the given event is present in weird_do_not_ignore_repeats. Events with notice-able actions are always recorded to weird_file.

• report_weird_conn (t: time, name: string, id: string, c: connection) Processes an occurrence of the weird event name associated with the connection c, which is described by the string id.

If report_weird_conn finds one of the hosts and the given event name in weird_ignore_host, then it does nothing. Then, if the event is in weird_action, then it looks up the event in weird_action_filters and invokes the corresponding function if present, otherwise taking the action from weird_action. It then implements the various flavors of WEIRD_NOTICE_xxx by not generating notices more than once per connection, originator host, etc., though the events are still written to weird_file. Finally, the function invokes to do the actual recording and/or writing to weird_file.

• report_weird_orig (t: time, name: string, id: string, orig: addr) Processes an occurrence of the weird event name associated with the source address orig. id textually describes the flow from orig to the destination, for example using endpoint_id.

The function looks up the event name in weird_action and passes it along to report_weird.

Events handled by conn_weird

• conn_weird (name: string, c: connection) Invoked for most "weird" events. name is the name of the weird event, and c is the connection with which it's associated.

conn_weird handles the following events, all of which have a default action of WEIRD_FILE:

• active_connection_reuse A new connection attempt (initial SYN) was seen for an already-established connection that has not yet terminated.
• bad_HTTP_reply The first line of a reply from an HTTP server did not include HTTP/ "version".
• bad_HTTP_version The first line of a request from an HTTP client did not include HTTP/version.
• bad_ICMP_checksum The checksum field in an ICMP packet was invalid.
• bad_rlogin_prolog The beginning of an Rlogin connection had a syntactical error.
• bad_RPC A Remote Procedure Call was ill-formed.
• bad_RPC_program A portmapper RPC call did not include the correct portmapper program number.
• bad_SYN_ack A TCP SYN acknowledgment (SYN-ack) did not acknowledge the sequence number sent in the initial SYN.
• bad_TCP_checksum A TCP packet had a bad checksum.
• bad_UDP_checksum A UDP packet had a bad checksum.
• baroque_SYN A TCP SYN was seen with an unlikely combination of other flags (the URGent pointer).
• blank_in_HTTP_request The URL in an HTTP request includes an embedded blank.
• connection_originator_SYN_ack A TCP endpoint that originated a connection by sending a SYN followed this up by sending a SYN-ack. * data_after_reset After a TCP endpoint sent a RST to terminate a connection, it sent some data.
• data_before_established Before the connection was fully established, a TCP endpoint sent some data.
• excessive_RPC_len An RPC record sent over a TCP connection exceeded 8 KB.
• excess_RPC The sender of an RPC request or reply included leftover data beyond what the RPC parameters or result value themselves consumed.
• FIN_advanced_last_seq A TCP endpoint retransmitted a FIN with a higher sequence number than previously.
• FIN_after_reset A TCP endpoint sent a FIN after sending a RST.
• FIN_storm The monitor saw a flurry of FIN packets all sent on the same connection. A ``flurry is defined as 1,000 packets that arrived with less than 1 sec between successive FINs.

Deficiency: Clearly, this numbers should be user-controllable.

• HTTP_unknown_method The method in an HTTP request was not GET, POST or HEAD.
• HTTP_version_mismatch A persistent HTTP connection sent a different version number for a subsequent item than it did initially.
• inappropriate_FIN A TCP endpoint sent a FIN before the connection was fully established.
• multiple_HTTP_request_elements An HTTP request included multiple methods.
• multiple_RPCs A TCP RPC stream included more than one remote procedure call.
• NUL_in_line A NUL (ASCII 0) was seen in a text stream that is expected to be free of NULs.

Updateme: Currently, the only such stream is that associated with an FTP control connection.

• originator_RPC_reply The originator (and hence presumed client) of an RPC connection sent an RPC reply (either instead of a request, or in addition to a request).
• partial_finger_request When a Finger connection terminated, it included a final line of unanalyzed text because the text was not newline-terminated.
• partial_ftp_request When an FTP connection terminated, it included a final line of unanalyzed text because the text was not newline-terminated.
• partial_ident_request When an IDENT connection terminated, it included a final line of unanalyzed text because the text was not newline-terminated.
• partial_portmapper_request A portmapper connection terminated with an unanalyzed request because the data stream was incomplete.
• partial_RPC An RPC was missing some required header information due to truncation.
• pending_data_when_closed A TCP connection closed even though not all of the data in it was analyzed due to a sequence hole.
• possible_split_routing Bro appears to be seeing only one direction of some bi-directional connections . This can also occur due to certain forms of stealth-scanning.
• premature_connection_reuse A TCP connection tuple is being reused less than 30 sec after its previous use. (The standard requires waiting 2 * @w{MSL} = 4 minutes [p. 27] [RFC-793].)
• repeated_SYN_reply_wo_ack A TCP responder that replied to an initial SYN with a SYN-ack has subsequently sent a SYN without an acknowledgment. * repeated_SYN_with_ack A TCP originator that sent an initial SYN has subsequently sent a SYN-ack.
• responder_RPC_call The responder (and hence presumed server) of an RPC connection sent an RPC request (either instead of a reply, or in addition to a reply).
• rlogin_text_after_rejected An Rlogin client sent additional text to an Rlogin server after the server already presumably rejected the client's service request.
• RPC_rexmit_inconsistency An RPC call was retransmitted, and the retransmitted call differed from the original call. This could reflect an attempt by an attacker to evade the monitor.

Note: This type of inconsistency checking is not available for RPC replies because the transmission of the reply in general marks the end of the RPC connection, and the monitor deletes the connection state shortly afterward.

• RST_storm The monitor saw a flurry of RST packets all sent on the same connection. See FIN_storm for the definition of ``flurry. * RST_with_data A TCP RST packet included data. This actually is allowed by the specification [4.2.2.12] RFC-1122. Deficiency: This event should include the data.
• simultaneous_open The monitor saw a TCP simultaneous open, i.e., both endpoints sent initial SYNs to one another at the same time. While the specification allows this [p. 30] RFC-793, none of the protocols analyzed by Bro should be using it.
• spontaneous_FIN A TCP endpoint sent a FIN packet without sending any previous packets. This event can reflect stealth-scanning, but can also occur when Bro has recently started up and has not seen other traffic on a connection and hence does not know that the connection already exists.
• spontaneous_RST A TCP endpoint sent a RST packet without sending any previous packets. As with spontaneous_FIN, this event can reflect either stealth scanning or a Bro start-up transient.
• SYN_after_close A TCP endpoint sent a SYN (connection initiation) after sending a FIN (connection termination), but before the connection fully closed. * SYN_after_partial A TCP endpoint in a "partial" connection sent a SYN.
• SYN_after_reset A TCP endpoint sent a SYN after sending a RST (reset connection).
• SYN_inside_connection A TCP endpoint sent a SYN during a connection (or partial connection) on which it had already sent data. * SYN_seq_jump A TCP endpoint retransmitted a SYN or a SYN-ack, but with a different sequence number.
• SYN_with_data A TCP endpoint included data in a SYN packet it sent. Note, this can legitimately occur for T/TCP connections [RFC-1644].
• TCP_christmas A TCP endpoint sent a SYN packet that included the RST flag (a nonsensical combination). The term "Christmas packet" has been used in this context (particularly if other flags are set, too) because the packet's flags are "lit up like a Christmas tree."
• UDP_datagram_length_mismatch The length field in a UDP header did not match the length field in the IP header. This could reflect an attempt by an attacker to evade the monitor.
• unpaired_RPC_response An RPC reply was seen for which no request was seen. This event could reflect a Bro start-up transient (it started running after the request was sent).
• unsolicited_SYN_response A TCP endpoint sent a SYN-ack without first receiving an initial SYN. This event could reflect a Bro start-up transient.

Events handled by conn_weird_addl

• conn_weird_addl (name: string, c: connection, addl: string) Invoked for a few ``weird events that require an extra (string) argument to help clarify the event. Deficiency: It would likely be very handy if the general ``weird event handling was more flexible, with the ability to have various parameters associated with the events. Doing so will likely have to wait on general Bro mechanism for dealing with default parameters and/or polymorphic functions and event handlers.

conn_weird_addl handles the following events, all of which have a default action of WEIRD_FILE:

• bad_ident_reply A reply from an IDENT server was syntactically invalid. * bad_ident_request A request to an IDENT server was syntactically invalid. * ident_request_addendum An IDENT request included additional text beyond that forming the request itself.

Events handled by flow_weird

• flow_weird (name: string, src: addr, dst: addr) is invoked for "weird" events that cannot be associated with a particular connection, but only with a pair of hosts, corresponding to a flow of packets from src to dst. Presently, all of these events deal with fragments.

flow_weird handles the following events:

• excessively_large_fragment A set of IP fragments reassembled to a maximum size exceeding 64,000 bytes.

Note: Sizes between 64,000 and 65,535 bytes are allowed, strictly speaking, but are highly unlikely in legitimate traffic. Sizes above 65,535 bytes generally represent attempted denial-of-service attacks, due to IP implementations that crash upon receiving such impossibly-large fragment sets.

Default: WEIRD_NOTICE_ALWAYS.

• excessively_small_fragment A fragment other than the last fragment in a set was less than 64 bytes in size.

Note: The standard allows such small fragments, but their presence may reflect an attacker attempting to evade the monitor by splitting header information across multiple fragments.

Default: WEIRD_NOTICE_ALWAYS.

• fragment_inconsistency A fragment overlaps with a previously sent fragment, and the two disagree on data they share in common. This event could reflect an attacker attempting to evade the monitor; it can also occur because Bro keeps previous fragments indefinitely, and occasionally a fragment will overlap with one sent much earlier and long-since forgotten by the endpoints.

Deficiency: Bro needs to provide a means for flushing old fragments, otherwise it becomes vulnerable to a state-holding attack.

Default: WEIRD_NOTICE_ALWAYS.

• fragment_overlap A fragment overlaps with a previously sent fragment. As for fragment_inconsistency, this event can occur due to Bro keeping previous fragments indefinitely. This event does not in general reflect a possible attempt at evasion.

Default: WEIRD_NOTICE_ALWAYS.

• fragment_protocol_inconsistency Two fragments were seen for the same flow and IP ID which differed in their transport protocol (e.g., UDP, TCP). According to the specification, this is allowed [p. 24] RFC-791, but its use appears highly unlikely.

Default: WEIRD_FILE, because it is difficult to see how an attacker can exploit this anomaly.

• fragment_size_inconsistency A ``last fragment was seen twice, and the two disagree on how large the reassembled datagram should be. This event could reflect an attacker attempting to evade the monitor.

Default: WEIRD_FILE, since it is more likely that this occurs due to a high volume flow of fragments wrapping the IP ID space than due to an actual attack.

• fragment_with_DF A fragment was seen with the "Don't Fragment" bit set in its header. While strictly speaking this is not illegal, and not impossible (a router could have fragmented a packet and then decided that the fragments should not be further fragmented), its presence is highly unusual.

Default: WEIRD_FILE, because it's difficult to see how this could reflect malicious activity.

• incompletely_captured_fragment A fragment was seen whose length field is larger than the fragment datagram appearing on the monitored link.

Default: WEIRD_NOTICE_ALWAYS.

Events handled by net_weird

• net_weird (name: string) is invoked for ``weird events that cannot be associated with a particular connection or set of hosts. Except as noted, the default action for all such events is WEIRD_FILE.

net_weird handles the following events:

• bad_IP_checksum A packet had a bad IP header checksum.

• bad_TCP_header_len The length of the TCP header (which is itself specified in the header) was smaller than the minimum allowed size.

• internally_truncated_header A captured packet with a valid IP length field was smaller as actually recorded, such that the captured version of the packet was illegally small. This event may reflect an error in Bro's packet capture hardware or software.

Default: WEIRD_NOTICE_ALWAYS, because this event can indicate a basic problem with Bro's packet capture.

• truncated_IP A captured packet either was too small to include a minimal IP header, or the full length as recorded by the packet capture library was smaller than the length as indicated by the IP header.

• truncated_header An IP datagram's header indicates a length smaller than that required for the indicated transport type (TCP, UDP, ICMP).

Events generated by the standard scripts

The following events are generated by the standard scripts themselves:

• bad_pm_port See pm_bad_port. Handled by conn_weird_addl, where the extra parameter is the text "port <bad-port>".

Land_attack A TCP connection attempt was seen with identical initiator and responder addresses and ports. This event likely reflects an attempted denial-of-service attack known as a ``Land attack. See check_spoof. Handled by conn_weird.

Additional handlers for ``weird events

In addition to the above, generalized events, Bro includes two specific events that are defined by themselves so they can include additional parameterization:

• rexmit_inconsistency (c: connection, t1: string, t2: string) Invoked when a retransmission associated with connection c differed in its data from the contents transmitted previously. t1 gives the original data and t2 the different retransmitted data.

This event may reflect an attacker attempting to evade the monitor. Unfortunately, however, experience has shown that (i) inconsistent retransmissions do in fact happen due to (appalling) TCP implementation bugs, and (ii) once they occur, they tend to cascade, because often the source of the bug is that the two endpoints have become desynchronized.

The handler logs the message in the format "id rexmit inconsistency (<t1>) (<t2>)" . However, the handler only logs the first instance of an inconsistency, due to the cascade problem mentioned above.

Deficiency: The handler is not told which of the two connection endpoints was the faulty transmitter.

• ack_above_hole (c: connection, t1: string, t2: string) Invoked when Bro sees a TCP receiver acknowledge data above a sequence hole. In principle, this should never occur. Its presence generally means one of two things: (i) a TCP implementation with an appalling bug (these definitely exist), or (ii) a packet drop by Bro's packet capture facility, such that it never saw the data now being acknowledged.

Because of the seriousness of this latter possibility, the handler logs a message ack above a hole. Note: You can often distinguish between a truly broken TCP acknowledgment and Bro dropping packets by the fact that in the latter case you generally see a cluster of ack-above-a-hole messages among otherwise unrelated connections.

Deficiency: The handler is not told which of the two connection endpoints sent the acknowledgment.

The icmp Analyzer

not done.

The stepping Analyzer

not done.

The ssh-stepping Analysis Script

not done.

The backdoor Analyzer

not done.

The interconn Analyzer

not done.

Reference Manual Introduction | Getting Started | Values, Types, and Constants | Statements and Expressions Global and Local Variables | Predefined Variables and Functions | Analyzers and Events Signatures | Interactive Debugger | Missing Documentation | References User Manual