The Client-Server Model
A server is a process that provides services. A client is a process that uses services. The terms server and client are also casually used to refer to the hosts on which server processes and client processes run.
Applications that run client and server processes (or client-server processes) are the basis of networking. Web services are provided using client-server processes. File downloads require client-server processes. Voice-Over-IP applications use client-server processes. There are innumerable client-server processes - everything from well-known applications such as telnet to applications written internally by organizations. Servers may provide access to devices such as printers, disks and storage arrays or to software services such as name services or file systems. Only network configuration and testing commands such as snoop or ping use the network but do not run client-server processes. Hosts on a network are commonly so interdependent that a system that provides services of one kind will probably be a client for other services.
Client-server processes can be cross-platform. Most of the well-known services such as ftp and http will run between hosts running different operating systems.
Review of Solaris
Management Facility (SMF)
In Solaris 10, services are managed by the Service Management Facility using svcadm and also inetadm.
Changes from Previous
Versions of Solaris
The biggest change in Solaris 10 is the adoption of the Service Management Facility (SMF) to take the place of the start scripts and the file /etc/inittab and also to take the place of the services file /etc/inetd.conf. Most services are no longer started and stopped using start and stop scripts, though many of those scripts remain available for backwards compatibility. The file /etc/inittab is used only for starting the most basic services and the new services daemon, svc.startd, which handles services configuration for the SMF. The role of the daemon init is greatly reduced. It reads the inittab, establishes sockets, starts svc.startd, and its main work is done. The daemon svc.startd reads the file /var/svc/profile/generic_open.xml, then goes on to read the manifests directed by this file. It also does run some run scripts, and you CAN add run scripts as you have in the past. You will continue to shut down with init though you can also use svcadm and go to milestone single-user rather than do init S.
The SMF
Everything in the SMF is organized into “services”. The term “services” is confusing, because these are not necessarily network services, but may be basic OS processes like starting the cron daemon or mounting file systems. Some services are even groupings of services called “milestones.” Configuration files for services are xml files called “manifests” and are located in /var/svc/manifest under directories related to the type of service. Milestone manifests are located in /var/svc/manifest/milestone. Networking manifests are located in /var/svc/manifest/network. These manifests call boot scripts which “register” certain kinds of network services called “rpc services,” that only start on demand (discussed later) or start network services that run all the time (also called start-at-boot processes). That means that services are once registered by the daemon inetd. The daemon inetd continues to receive packets and direct them to the correct ports, and to start start-on-demand services.
“Milestone services” are just sets of services on which the milestone “depends.” When all the services in a milestone are successfully started, the milestone’s dependencies are met and the milestone is successfully completed. No additional services are started, separate from the milestone’s dependencies. It just means that the kernel does not consider the milestone service to be started until all dependent services are started. This makes the milestones more reliable than run scripts as a way of setting run states. With run scripts it was perfectly possible to mess up the boot so badly that nothing worked, because the only their names controlled the order in which run scripts started. Now the milestone manifests control services and you actually have to edit the files to change the boot process.
There is a group of three milestones which emulate the run levels S, 2 and 3. These are the milestones called single-user, multi-user and multi-user-server. You still change between them and perform shutdown using init. There are also milestones used to indicate that other services, dependent on that milestone can now be started, and that you, as administrator, will not need to change. These include “network,” which indicates that the physical network is configured, “name-services,” which indicates that configured name services are running, and “devices,” which runs when all devices are configured and functioning. The “telnet” application will not be started unless the “network” milestone is reached. File systems are likewise dependent on the “devices” milestone.
You can create and edit manifests, though it is unlikely that you would do so. More likely you will add manifests to /var/svc/manifest that are provided to you as part of software packages, and edit the profiles that tell them to start. There is an empty directory called “site” provided for local manifests.
When services are configured by the administrator on the command line, that information is kept in the “repository” which is a set of database files kept in /etc/svc/repository.db. older versions of the configuration are kept in other .db files in that directory, and the command svccfg can be used to back them out OR /lib/svc/bin/restore_repository. The initial repository is kept in /etc/svc/seed/global.db, and can be restored by running the command /lib/svc/bin/restore_repository and selecting the seed repository then rebooting.
SMF Commands:
svcs - prints out services and their states.
Service states:
legacy_run - services started by scripts (script names are listed)
online - service started
offline - service not started even though manifest has been run, but there has been no return code. The “listener” that controls printing won’t start if there is no printer configured.
disabled - deliberately disabled using command svcadm, below, or a dependency was disabled.
degraded - rare - might result from a typo in manifest.
maintenance - rare - temporarily down
The listing specifying the service is a Fault Management Resource Identifier or FMRI. We will use complete and partial FMRIs to indicate services we want to administer. You can use the entire FMRI in the command line, or a part of it. The FMRI consists of the location of the manifest under /var/svc/manifest, followed by a colon and the instance of the service, which is most commonly “default,” for the single default instance.
svcs -a - lists all services including those that are disabled.
svcs - lists non-disabled services
svcs (service name) - list info for that service
svcs -p (service name or string) - lists processes including PIDs associated with a service.
svcs -d (service name or string) - lists processes depending on the specified process.
svcs -D (service name or string) - lists processes on which the specified process depends.
svcs -l (specified process) lists all configuration info including dependencies.
svcs -x find out why a service isn’t running - CHECK OUT
svcadm - allows you to modify services. Also allows you to change milestones.
This can set permanent changes in configuration at the command line, without modifying files. If you disable finger using:
svcadm disable finger
That outlasts a reboot just as though you had edited the file /etc/inetd.conf.
This command requires you to use the FMRI or a unique fraction thereof to describe the service you want to modify. The list of subcommands are in the book on p 4-31. To enable a service:
svcadm enable finger
To enable a service and all its dependencies
svcadm -r enable finger (though finger has no dependencies)
To change milestone:
svcadm milestone
single-user
inetadm
This command allows you to observe and configure inetd controlled services.
inetadm - shows all services and their states
inetadm -l shows properties (essentially what used to be in /etc/inetd.conf).
inetadm -e enables properties
inetadm -d disables properties
So inetadm -d finger does what svcadm disable finger does.
inetadm -p shows all service properties.
Client-Server
Processes
The network services started or registered by the SMF adhere to a set of standards for TCP/IP (Transmission Control Protocol/Internet Protocol) networking that apply to all client-server processes. These standards, called “RFCs,” are written and controlled by an international committee, the Internet Engineering Task Force, or IETF. Sun Microsystems has no control over these standards, although as a major innovator and developer of networking software, it will usually have delegates on some standards committees. Vendors of networking software can choose which RFC standards to implement, but the Internet uses TCP/IP networking, so pretty much all networking software vendors do write software that adheres to the RFCs. There is a lot of room in the standards to innovate, so each vendor will implement the standards differently, but the essential concepts, and the results must be the same (or the vendor will not sell any software!). One requirement is that implementations be cross-platform. For example, “telnet” on an Solaris computer can be used to connect to a PC running Windows. The “telnet” application on that PC can then be used to connect to a Linux computer. That ability to connect cross-platform is part of RFCs for any application. Vendors are also free to write software that adheres to no RFC, and this kind of software commonly forms the basis for later RFCs. Such software cannot be used cross-platform, since no one but the innovating vendor is writing the software, and it is unlikely that anyone else will adopt it. Sun Microsystems has a number of applications that use TCP/IP networking standards, but that are not specified by any RFC. These applications can be used only by computers running the Solaris OS and are useless in multi-platform networks. As a result, such applications are usually limited to software such as management GUIs that are not essential to the function of the system, or when Sun does not intend to allow cross-platform use of their software, as in the case of Sun Cluster software.
Sun Microsystems implements client-server processes using a daemon called ‘inetd.”
inetd: inetd is a daemon that monitors the network connection for requests for services to be started on demand, and starts those services as they are requested. These services may be of two kinds: "well known" services, and rpc services. Well-known services are those services which are widely used and cross-platform. Each of these services is always assigned the same, well-known, "port number" (one of 1-1024), such as port 23 for telnet. A port number is simply a numerical reference to an application that implements client-server processes. "Well known" services are listed in /etc/services by their assigned port numbers, which are controlled by ICANN (Internet Corporation for Assigned Names and Numbers).
RPC: Rpc services are less well known; they may be Sun software or widely available Unix utilities; but they may also be software from a small company, or even software written at the site at which it runs. Rpc services do not have assigned port numbers, so at boot time, the kernel assigns an “ephemeral” or “dynamic” port number to each of these programs, thus "registering" that service. Processes that start at boot from start scripts request port number assignment from the kernel directly. Programs that start on demand are registered by the daemon inetd when they start. These ports are also said to be “dynamically bound” to the service, which just means that an ephemeral port number was assigned to the program at boot. Thus every program that can be started over the network has a port number associated with it by the time boot has finished. Well known services get the same number every time. Rpc services may get different numbers every time. Rpc services are listed in /etc/rpc by program number, and their program number is associated with its ephemeral port number in a table in RAM that is set up at boot time and monitored by a daemon called rpcbind. It is possible for more than one process to list the same port number: it just means those processes employ the same application.
Program numbers are assigned by IANA as of about 2004, but were previously controlled by Sun Microsystems, who invented what is now the RPC standard.
Starting services: Programs requested over the network, both well-known and rpc, are started in one of two ways. They may be started at boot time, from methods. The method /usr/lib/svc/method/smtp-sendmail starts sendmail which is a well-known start-at-boot service. There are also rpc start-at-boot services such as mountd. Services that start at boot request their own port numbers from the kernel. From that time on, the service monitors its own port number and answers service requests. Services such as telnet (well known) or rusers (rpc) will be started on-demand by the daemon inetd when requested. A program which must be started on demand, whether it is well known or rpc, must be registered by the SMF at boot time. The SMF also provides the startup information necessary for inetd to start the program, including the full path to the command and all options.
The client-server
process
On the client system, when a request for a well-known program is issued by a user, the kernel assigns the outgoing request a random port for the source port and uses the well known port number as the destination port. When that request for a well known service arrives at the server, inetd on the server looks in /etc/inet/services to find the name of the program associated with that port, then uses the information from the SMF to start the appropriate daemon, which then provides the service (if the processes starts on request) or passes control of the service to an already-running program (if the process starts at boot time). For example, if inetd gets a request on port 23, it looks in /etc/services and sees that port 23 is associated with telnet. It looks for information on telnet in the SMF and uses that to start the daemon in.telnetd. A socket is then created, and takes over control of the process. Packets are sent between the well known port on the server and the randomly assigned port on the client.
This only works for well known services, like ftp or telnet, which always are assigned the same ports on every operating system. For rpc services the client host sends a request with a program number from the file /etc/rpc to the destination host on port 111, which
is the port associated with the daemon rpcbind. Inetd passes the request to rpcbind, which looks in the file /etc/rpc for the program name and program (NOT port!) number, then checks the table in RAM to see if there is a port number mapped to that program number. If there is, rpcbind passes the port number back to inetd, which sends the associated port number back to the client computer, which then sends a request for the service to that port. Inetd then starts the appropriate service or passes control of the service to an already-running program (if the process starts at boot time).
Preventing services from starting: You can prevent services from being offered on your system (finger!) by disabling them in the SMF. This does not prevent you from requesting services, since the kernel assigns a random port to the outgoing request, not a specified port. It simply prevents your system from responding to services.
Practical Uses of Port
Numbers
These detailed internals of port number function may seem irrelevant to everyday management of a network, since almost all of it happens automatically and requires no configuration. In reality, understanding port numbers is crucial in handling security for systems connected to the Internet. For example, a group of local area networks under the control of one authority is called an Autonomous System. The Autonomous System will be connected to the internet via a heavily secured computer called a gateway, which is any system that transmits packets between the Internet and an Autonomous System. The gateway may be configured to accept only certain packets from the Internet with firewalling software. Typically firewalls are set up so that any packets directed to port 23 are discarded – thus users within the Autonomous System can telnet to each other, but never outside the Autonomous System, and no one outside can telnet in. In this example, packets used for ftp can still move in and out of the Autonomous System. Can you firewall RPC processes the same way? No! RPC processes do not use the same port every time! Firewalling only specific RPC processes is going to be much more complex as a result and will require different firewall rules than well-known processes do. The foregoing is just one example of why it is critical to have a good grasp of networking details in order to secure your networks.
Port numbers are also used in setting up NAT (Network Address Translation) for private networks. When a host on the private network generates a packet, the gateway picks it up, and adds the source port and the private IP address of the originating system to a “NAT table” along with a port number chosen by the gateway. The packet is then repackaged with the IP address of the gateway as its source IP and the port number assigned to the system in the gateway’s NAT table as its source port number. When a packet is sent in return, the gateway then uses the destination port number, along with its NAT table, to direct the packet to the correct port on the correct originating host.
Monitoring services: rpcinfo is used to manage the rcp services. With no options, it
identifies the rpc programs registered on the local host. With
Acronyms:
IANA – Internet Assigned Numbers Authority – the organization that oversees IP address and port number uses, under the direction of ICANN.
ICANN – Internet Corporation for Assigned Names and Numbers. A new, non-profit, international organization that oversees assignment of IP addresses and of domain names.
IETF - Internet Engineering Task Force - the organization that writes and publishes all networking standards for the TCP/IP networking model.
RFC - Request For Comments - TCP/IP standards. The name is historical, and despite its inclusive sound, the IETF really does not want to hear from us.
RPC – Remote Procedure Call – a program which calls on the counterpart program on another host.
TCP/IP -Transmission Control Protocol/Internet Protocol - a set of standards (the RFCs). All together the RFCs form a network model. The Internet is a TCP/IP network, but other network models are occasionally implemented on dedicated networks. The now-defunct AppleTalk network was an ISO/OSI network.
TI-RPC – Transport Independent Remote Procedure Call – The type of RPC protocols used by SVR4. They allow applications written with this protocol to run on any Operating system, so that applications can exchange data across heterogenous environments.
Definitions:
arbitrary/ephemeral port – a port of number greater than or equal to 32768, which the kernel randomly assigns to an exiting datagram generated by client process, or which is assigned to an rpc server.
port – an address space in memory: designates the program resident at that address space. The port’s number is used to direct incoming packets to the correct program.
privileged – ports from 1 to 1024 which provide the "well known services." Root owns all these processes.
rpc call – a request to run a specific rpc application. The kernel on the client sends a request to the kernel on the server.
rpc services – services which are not "well known", and which are started via the daemon rpcbind (port 111). They have the format rpc.xxx (where xxx) is some number) in the file /etc/inetd.conf, and are listed in /etc/rpc. They are assigned "ephemeral" ports at boot time (those greater than 32768).
socket– BSD interface between the transport layer and a program.
"true" environment – one in which only the services requested are started.
"well known" services – published, established, widely used services, registered with ICANN and assigned a “well known” port number. Some of these services are actually so obsolete that no one implements them any more, so this is more of a functional definition than a practical one. Many “well known” services aren’t well known at all, but if they have assigned port numbers, they belong to this grouping.
Commands:
rpcinfo – RPC information and management utility.
no options - identify the rpc programs registered on the local host.
-p <host> - identify the rpc services registered on the host listed. Output includes
the registered port for the service.
-b <program> <version> - identify servers on the network running the
specified version of the specified program.
-u <server> <program> - identify all versions of the specified program running
on the specified server.
-d <program> <version> - unregister the specified version of the specified
program.
-t <server> shows rpc information for tcp programs.
netstat -a - shows established connections and ports in use. Swind is the send buffer size, and Rwind is the advertised reception window size.
Files:
/etc/inet/services – contains the well known services. Format: service name port/protocol
service name – the name of the service
port – the well known port number assigned to that service by IANA.
protocol – the protocol, such as tcp or udp used by the service.
/etc/rpc – contains the rpc services. Format: service name program number aliases
service name – the name of the service
program number – the number assigned to the program
aliases – other names by which the service is known.
Misc:
Properties of a service:
service name – service listed in /etc/services OR /etc/rpc, in the format
name|program number/version for rpc services.
endpoint: socket type (stream, dgram, raw, seqpacket) or tli.
protocol: from /etc/inet/protocols (for well known services). For rpc, the value is "rpc/*" or "rpc/<netid>," where the list implied by * is found in the file /etc/netconfig.
wait status: "wait" means: if a program is requested, but is already running, wait til the first one stops running to start the next. "no wait"means start multiple instances. UDP services must all be "no wait;" TCP can be "no wait."
uid – the user identification number to use as the process owner. For "well known" services, this value must be 0 (root), and thus they are described as "privileged."
program – if inetd provides the service, the word "internal". Otherwise the absolute path to the command that provides the service requested.
args –the command plus all options.
Protocol stack for rpc applications ISO/OSI layer
XDR presentation
TI-RPC session
TLI-API (SVR4)/Sockets API(BSD) session
TCP/UDP transport
IP internet
Services started at boot time by start scripts:
sendmail,
Services started on demand by /etc/inetd.conf:
telnet, ftp, wall, rup