Single Client Access Name (SCAN) ---------------------------------------------------------------------------------------------------- 11gR2 RAC feature General description at http://www.oracle.com/technetwork/database/clustering/overview/scan-129069.pdf http://download.oracle.com/docs/cd/E11882_01/install.112/e10812/typinstl.htm#BABDFFDG ---------------------------------------------------------------------------------------------------- * SCAN can mean scan VIP or scan listener `srvctl -h | grep scan' shows two sets of commands: `srvctl xxx scan' for scan VIP management and `srvctl xxx scan_listener' for scan listener. `crs_stat -t | grep -i scan' most likely only shows scan VIP's due to string abbreviation. You can use my crs_stat.sh to replace `crs_stat -t': $ crs_stat -t | grep -i scan ora.scan1.vip ora....ip.type ONLINE ONLINE host2 ora.scan2.vip ora....ip.type ONLINE ONLINE host1 ora.scan3.vip ora....ip.type ONLINE ONLINE host1 $ cat crs_stat.sh #!/bin/bash #crs_stat.sh: a replacement for `crs_stat -t', showing full CRS component name with its state on one line echo "Columns are: Name Target State Host" crs_stat | perl -ne 'chomp;if(/^NAME=(.*)/){print $1}elsif(/^TARGET=(.*)/){print "\t$1"}elsif(/^STATE=(.*)/){print "\t$1\n"}' $ crs_stat.sh | grep -i scan ora.LISTENER_SCAN1.lsnr ONLINE ONLINE on host2 ora.LISTENER_SCAN2.lsnr ONLINE ONLINE on host1 ora.LISTENER_SCAN3.lsnr ONLINE ONLINE on host1 ora.scan1.vip ONLINE ONLINE on host2 ora.scan2.vip ONLINE ONLINE on host1 ora.scan3.vip ONLINE ONLINE on host1 ---------------------------------------------------------------------------------------------------- * Tracing SCAN listener Not much difference from tracing a regular listener: $ ORACLE_HOME=/u01/app/11.2.0/grid $ORACLE_HOME/bin/lsnrctl LSNRCTL> set current_listener LISTENER_SCAN1 Current Listener is LISTENER_SCAN1 LSNRCTL> set trc_level 16 Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=LISTENER_SCAN1))) LISTENER_SCAN1 parameter "trc_level" set to support The command completed successfully LSNRCTL> set trc_level 0 Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=LISTENER_SCAN1))) LISTENER_SCAN1 parameter "trc_level" set to off The command completed successfully Both the trace and log file (e.g. listener_scan1.log) are under $/log/diag/tnslsnr/`hostname -s`/listener_scan?/trace $ vi /u01/app/11.2.0/grid/log/diag/tnslsnr/`hostname -s`/listener_scan1/trace/ora_*.trc $ egrep 'load|what:[24]' The above egrep command views node and instance load stats (Ref: Note:263599.1). "what:4" shows delta and "what:2" shows the so-called goodness value (spelled "goodnes" in `oradebug dump listener_registration '). Actually, trc_level 4 (user) is enough to show info about load stats. Other levels are: 0,1: off 2,3: err 4,5: user 6-14:admin 15: dev 16-: support ---------------------------------------------------------------------------------------------------- * Find SCAN name on DB server SCAN hostname is not in /etc/hosts, nor is it `hostname`. To find it, $ /sbin/ifconfig | grep "inet addr" inet addr:10.112.50.79 Bcast:10.112.50.255 Mask:255.255.255.0 inet addr:10.112.50.100 Bcast:10.112.50.255 Mask:255.255.255.0 inet addr:10.112.50.103 Bcast:10.112.50.255 Mask:255.255.255.0 ... $ host 10.112.50.103 103.50.112.10.in-addr.arpa domain name pointer scan.example.com. Command `host' can also be `nslookup'. If the SCAN listener is up, of course you can also find it by netstat -anp | grep LISTEN host ---------------------------------------------------------------------------------------------------- * Find real hostname by SCAN from client side If SCAN hostname (e.g. scan.example.com) significantly differs from the real hostname (e.g. dbhost.example.com), it's hard to guess the real hostname. You can ssh to the host with SCAN and use `hostname' or equivalent to find the real hostname. If you don't have a shell account on the host, use sqlplus to connect to the database running on the SCAN host and query v$instance. If you have neither access, just type sqlplus a/b@conn, where a and b are any strings, conn is a connection string that uses this SCAN (create an entry in local tnsnames.ora if not exists, or use full TNS specification). Then check the remote address of this connection with netstat -a | findstr 1521 (on Windows, assuming 1521 is the port) Tcpview (on Windows) netstat -a | grep 1521 (on *NIX) There should be two lines, e.g.: c:\>netstat -a | findstr 1521 TCP mylaptop:2335 scan.example.com:1521 TIME_WAIT TCP mylaptop:2336 dbhostvip.example.com:1521 ESTABLISHED The remote address for the ESTABLISHED line is the VIP of the DB server. By the way, the TIME_WAIT line indicates a recent past connection from mylaptop to the scan listener, which immediately handed off to the real listener on the database server. They can both use the same port because there's no conflict in the unique combination of {host, port, transport protocol}. Obviously, this method only works if your local workstation or laptop has not made other Oracle Net connections, or those connections would interfere. Run `netstat -a | findstr 1521' first and make sure there's no output. ---------------------------------------------------------------------------------------------------- * Problem with pre-11gR2 clients https://support.oracle.com/CSP/main/article?cmd=show&type=NOT&id=1188736.1 "The older version Oracle [pre-11g Release 2] Client will not be able to handle a set of three IPs returned by the DNS for SCAN. Hence, it will try to connect to only the first address returned in the list and will more or less ignore the others. If the SCAN Listener listening on this specific IP is not available or the IP itself is not available, the connection will fail." The suggested workaround is to write the three IP's the SCAN resolves to into tnsnames.ora. Here's an easy way to verify. Do it on the client (replace strace with truss as needed). nslookup scan #or `host scan'; scan is our scan.example.com; run multiple times to make sure they rotate ping scan #run multiple times to make sure the client connects to only 1 IP strace tnsping connectionstring 2>&1 | grep #use TCPView instead on Windows If you run the last command using 11gR2 client, you should see 3 IP's. Using an older client, you see only 1. If you don't use the suggested workaround (and hope most clients will be 11gR2 soon), when one IP resolved by scan goes bad and is not restarted by CRS for some reason, ask clients to flush client side DNS: ipconfig/flushdns. He will get hooked to a good surviving IP. ---------------------------------------------------------------------------------------------------- * Overhead (or not) The extra listeners in 11gR2, called SCAN listeners, won't use much CPU or memory. But there's another aspect of overhead. They hand off connections to the regular listeners. If I connect to the regular listener in a dedicated connection, the listener fork()'s (clone()'s in Linux) and exec()'s $ORACLE_HOME/bin/oracle. Do you think the SCAN listener does the same if the least loaded node happens to be the current node? No. It's still the regular listener that does that. This can be verified with simple strace. On a 2-node RAC, 50% of the time this extra hand-off would be made unnecessary if the SCAN listener could spawn a server process. If the database has frequent client connections and disconnections, this overhead may be non-negligible. The SCAN listeners also have load stats of all the nodes. The probability of node-to-node redirection is the same as using the traditional VIPs in the connection strings, even if you have more than 3 nodes. (Hope I'm getting this right.) And the regular listeners shouldn't need to store each node's load stats. There'll be another part of overhead if the regular listener is not modified to skip checking load stats. (It probably still checks, based on the fact that it still has pmon's service update as the SCAN listener does.) ---------------------------------------------------------------------------------------------------- Something else is different in 11gR2. Both types of listeners open and then close their log files frequently, unlike in previous versions where the listener process holds the log file open all the time. If they very frequently open and close the files, that may be a performance problem. ---------------------------------------------------------------------------------------------------- Both regular and SCAN listeners can use port 1521, because they use different hostnames. (Only the combination of {hostname, port, protocol} is required by the transport layer network protocol to be unique.) If you do decide to change SCAN listener port, run `srvctl modify scan_listener -p ', stop and start it (srvctl (stop|start) scan_listener), and alter system set remote_listener=':'. ---------------------------------------------------------------------------------------------------- Using SCAN address won't magically correct the problem that some ODBC drivers don't support failover. For example, sessions using Microsoft ODBC driver for Oracle still have NONE under failover_(method|type) in v$session even if you use SCAN in the connection string. (Ref: http://www.freelists.org/post/oracle-l/Not-to-use-SCAN-in-11gR2-RAC) ---------------------------------------------------------------------------------------------------- Service does not register References: 11gR2 PMON Issue: Services Fail to Register to SCAN Listeners [ID 1448717.1] How to Trace Dynamic Registration from PMON ? [ID 787055.1] If (oraenv set to ASM) lsnrctl service LISTENER_SCANx shows no services, login to the database and alter system set events='immediate trace name listener_registration level 3'; pmon trace file has nse (probably network service error) lines, e.g. 3 - (DESCRIPTION=(CONNECT_DATA=(SERVICE_NAME=))(ADDRESS=(PROTOCOL=TCP)(HOST=10.114.177.33)(PORT=1521))) state=4, err=3 nse[0]=12541, nse[1]=12541, nte[0]=0, nte[1]=0, nte[2]=0 ncre=0 endp=(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=10.114.177.33)(PORT=1521))) flg=0x80000000 nse=0 TNS-12541 means "TNS:no listener". Find on which node 10.114.177.33 is. This SCAN listener may be running on a node other than the one with that IP. Stop the listener and start it explicitly specifying the correct node: srvctl stop scan_listener -i x srvctl start scan_listener -i x -n nodename ----------------------------------------------------------------------------------------------------