Single Client Access Name (SCAN) ---------------------------------------------------------------------------------------------------- 11gR2 RAC feature General description at http://www.oracle.com/technetwork/database/clustering/overview/scan-129069.pdf http://download.oracle.com/docs/cd/E11882_01/install.112/e10812/typinstl.htm#BABDFFDG ---------------------------------------------------------------------------------------------------- * Overhead of SCAN The extra listeners in 11gR2, called SCAN listeners, won't use much CPU or memory. But there's another aspect of overhead: they hand off connections to the regular listeners. If I connect to the regular listener in a dedicated connection, the listener fork()'s (clone()'s in Linux) and exec()'s $ORACLE_HOME/bin/oracle. Does the SCAN listener do the same if the least loaded node happens to be the current node? No. It's still the regular listener that does that. This can be verified with simple strace. Thus, on a 2-node RAC, 50% of the time this extra hand-off would be made unnecessary if the SCAN listener itself could spawn a server process. If the database has frequent client connections and disconnections, this overhead may be non-negligible. The SCAN listeners also collect load stats of all the nodes. But the regular listeners also do, as usual. You can see PMON's service update in both SCAN and regular listeners' log files. This extra load stats collection adds another part of overhead. ---------------------------------------------------------------------------------------------------- * Overhead of DNS Connecting to SCAN starts with consulting with DNS. Some clients have DNS cache, which relieves pressure on the DNS server, but they may be disabled in some shops to avoid bugs. If DNS does not perform well and a client connects to the database very frequently, this extra DNS lookup adds a non-negligible delay to the end user's experience. If the firewall sits between the client and the RAC database, not using SCAN reduces 3 IP's to be open on the firewall, which is generally welcomed by the security personnel. ---------------------------------------------------------------------------------------------------- Benefits of SCAN Less typing in a TNS entry. If you add or remove a node, or make a node unavailable or available, you don't need to modify all your clients' TNS entry. But many shops centrally manage TNS (by periodically distributing a standard tnsnames.ora to all clients, placing the file on a shared drive, using OID, Oracle Names, etc). This benefit is minimized. Because SCAN is tied to 3 IPs, DNS provides round-robin on the client side. If you use VIP's without SCAN, you have to remember to add load_balance=yes to the TNS stanza to have the same effect. Some clients have trouble accepting more than one database host entry. SCAN is a life saver. If you use policy-managed databases, SCAN is mandatory. Installation of RAC requires SCAN. But you can choose to not use it later, or even disable it. ---------------------------------------------------------------------------------------------------- * SCAN can mean scan VIP or scan listener `srvctl -h | grep scan' shows two sets of commands: `srvctl xxx scan' for scan VIP management and `srvctl xxx scan_listener' for scan listener. `crs_stat -t | grep -i scan' most likely only shows scan VIP's due to string abbreviation. You can use my crs_stat.sh to replace `crs_stat -t': $ crs_stat -t | grep -i scan ora.scan1.vip ora....ip.type ONLINE ONLINE host2 ora.scan2.vip ora....ip.type ONLINE ONLINE host1 ora.scan3.vip ora....ip.type ONLINE ONLINE host1 $ cat crs_stat.sh #!/bin/bash #crs_stat.sh: a replacement for `crs_stat -t', showing full CRS component name with its state on one line echo "Columns are: Name Target State Host" crs_stat | perl -ne 'chomp;if(/^NAME=(.*)/){print $1}elsif(/^TARGET=(.*)/){print "\t$1"}elsif(/^STATE=(.*)/){print "\t$1\n"}' $ crs_stat.sh | grep -i scan ora.LISTENER_SCAN1.lsnr ONLINE ONLINE on host2 ora.LISTENER_SCAN2.lsnr ONLINE ONLINE on host1 ora.LISTENER_SCAN3.lsnr ONLINE ONLINE on host1 ora.scan1.vip ONLINE ONLINE on host2 ora.scan2.vip ONLINE ONLINE on host1 ora.scan3.vip ONLINE ONLINE on host1 ---------------------------------------------------------------------------------------------------- * Tracing SCAN listener Not much difference from tracing a regular listener: $ ORACLE_HOME=/u01/app/11.2.0/grid $ORACLE_HOME/bin/lsnrctl LSNRCTL> set current_listener LISTENER_SCAN1 Current Listener is LISTENER_SCAN1 LSNRCTL> set trc_level 16 Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=LISTENER_SCAN1))) LISTENER_SCAN1 parameter "trc_level" set to support The command completed successfully LSNRCTL> set trc_level 0 Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=LISTENER_SCAN1))) LISTENER_SCAN1 parameter "trc_level" set to off The command completed successfully Both the trace and log file (e.g. listener_scan1.log) are under $/log/diag/tnslsnr/`hostname -s`/listener_scan?/trace $ vi /u01/app/11.2.0/grid/log/diag/tnslsnr/`hostname -s`/listener_scan1/trace/ora_*.trc $ egrep 'load|what:[24]' The above egrep command views node and instance load stats (Ref: Note:263599.1). "what:4" shows delta and "what:2" shows the so-called goodness value (spelled "goodnes" in `oradebug dump listener_registration '), the higher the value, the less attractive this instance is to the service. Actually, trc_level 4 (user) is enough to show info about load stats. Other levels are: 0,1: off 2,3: err 4,5: user 6-14:admin 15: dev 16-: support By the way, you can also trace the listener by lsnrctl trace lsnrctl trace off See Note 147446.1. ---------------------------------------------------------------------------------------------------- * Find SCAN name on DB server SCAN hostname is not in /etc/hosts, nor is it `hostname`. To find it, $ /sbin/ifconfig | grep "inet addr" inet addr:10.112.50.79 Bcast:10.112.50.255 Mask:255.255.255.0 inet addr:10.112.50.100 Bcast:10.112.50.255 Mask:255.255.255.0 inet addr:10.112.50.103 Bcast:10.112.50.255 Mask:255.255.255.0 ... $ host 10.112.50.103 103.50.112.10.in-addr.arpa domain name pointer scan.example.com. Command `host' can also be `nslookup'. If the SCAN listener is up, of course you can also find it by netstat -anp | grep LISTEN host ---------------------------------------------------------------------------------------------------- * Find real hostname by SCAN from client side If SCAN hostname (e.g. scan.example.com) significantly differs from the real hostname (e.g. dbhost.example.com), it's hard to guess the real hostname. You can ssh to the host with SCAN and use `hostname' or equivalent to find the real hostname. If you don't have a shell account on the host, use sqlplus to connect to the database running on the SCAN host and query v$instance. If you have neither access, just type sqlplus a/b@conn, where a and b are any strings, conn is a connection string that uses this SCAN (create an entry in local tnsnames.ora if not exists, or use full TNS specification). Then check the remote address of this connection with netstat -a | findstr 1521 (on Windows, assuming 1521 is the port) Tcpview (on Windows) netstat -a | grep 1521 (on *NIX) There should be two lines, e.g.: c:\>netstat -a | findstr 1521 TCP mylaptop:2335 scan.example.com:1521 TIME_WAIT TCP mylaptop:2336 dbhostvip.example.com:1521 ESTABLISHED The remote address for the ESTABLISHED line is the VIP of the DB server. By the way, the TIME_WAIT line indicates a recent past connection from mylaptop to the scan listener, which immediately handed off to the real listener on the database server. They can both use the same port because there's no conflict in the unique combination of {host, port, transport protocol}. Obviously, this method only works if your local workstation or laptop has not made other Oracle Net connections, or those connections would interfere. Run `netstat -a | findstr 1521' first and make sure there's no output. ---------------------------------------------------------------------------------------------------- * Problem with pre-11gR2 clients https://support.oracle.com/CSP/main/article?cmd=show&type=NOT&id=1188736.1 "The older version Oracle [pre-11g Release 2] Client will not be able to handle a set of three IPs returned by the DNS for SCAN. Hence, it will try to connect to only the first address returned in the list and will more or less ignore the others. If the SCAN Listener listening on this specific IP is not available or the IP itself is not available, the connection will fail." The suggested workaround is to write the three IP's the SCAN resolves to into tnsnames.ora. Here's an easy way to verify. Do it on the client (replace strace with truss as needed). nslookup scan.example.com #or `host scan.example.com'; scan is our database SCAN "host" name; run multiple times to make sure they rotate ping scan.example.com #run multiple times to make sure the client connects to only 1 IP strace tnsping connectidentifier 2>&1 | grep #If on Windows, use TCPView instead If you run the last command using 11gR2 client, you should see 3 IP's. Using an older client, you see only 1. If you don't use the suggested workaround for 10g clients, when one IP resolved by scan.example.com goes bad and is not restarted by CRS for some reason, ask clients to flush client side DNS: ipconfig/flushdns. He will get hooked to a good surviving IP. ---------------------------------------------------------------------------------------------------- Something else is different in 11gR2. Both types of listeners open and then close their log files frequently, unlike in previous versions where the listener process holds the log file open all the time. If they very frequently open and close the files, that may be a performance problem. ---------------------------------------------------------------------------------------------------- Both regular and SCAN listeners can use port 1521, because they use different hostnames. (Only the combination of {hostname, port, protocol} is required by the transport layer network protocol to be unique.) If you do decide to change SCAN listener port, run `srvctl modify scan_listener -p ', stop and start it (srvctl (stop|start) scan_listener), and alter system set remote_listener=':'. ---------------------------------------------------------------------------------------------------- Using SCAN address won't magically correct the problem that some ODBC drivers don't support failover. For example, sessions using Microsoft ODBC driver for Oracle still have NONE under failover_(method|type) in v$session even if you use SCAN in the connection string. (Ref: http://www.freelists.org/post/oracle-l/Not-to-use-SCAN-in-11gR2-RAC) ---------------------------------------------------------------------------------------------------- Service does not register References: 11gR2 PMON Issue: Services Fail to Register to SCAN Listeners [ID 1448717.1] How to Trace Dynamic Registration from PMON ? [ID 787055.1] If (oraenv set to ASM) lsnrctl service LISTENER_SCANx shows no services, login to the database and alter system set events='immediate trace name listener_registration level 3'; pmon trace file has nse (probably network service error) lines, e.g. 3 - (DESCRIPTION=(CONNECT_DATA=(SERVICE_NAME=))(ADDRESS=(PROTOCOL=TCP)(HOST=10.114.177.33)(PORT=1521))) state=4, err=3 nse[0]=12541, nse[1]=12541, nte[0]=0, nte[1]=0, nte[2]=0 ncre=0 endp=(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=10.114.177.33)(PORT=1521))) flg=0x80000000 nse=0 TNS-12541 means "TNS:no listener". Find on which node 10.114.177.33 is. This SCAN listener may be running on a node other than the one with that IP. Stop the listener and start it explicitly specifying the correct node: srvctl stop scan_listener -i x srvctl start scan_listener -i x -n nodename ----------------------------------------------------------------------------------------------------