Internet Technology

My notes on all the protocols and formats that I’ve had to deal with. Important organizations are the Internet Engineering Task Force (IETF), the Internet Assigned Numbers Authority (IANA), and World Wide Web Consortium (W3C). Most of the protocols are standardized and kept at the IETF; some RFCs (Requests For Comment) also qualify as BCPs (Best Current Practices) or STDs (Internet Standards). See RFC 2026 (which is also BCP0009) for more information. When I list current RFCs, I also list their antecedents— I’ve often found that many products haven’t caught up to the latest standards yet. Hyperlinks in headers are to appropriate working groups or other important central locations for the particular standard. RFC2151, A Primer On Internet and TCP/IP Tools and Utilities may be of interest.

When I have to deal with a new standardized protocol or format, I go to the IETF and start hunting down all the RFCs for it and organize them into a location on a machine that’s on my LAN, to be accessed via a web server or just plain file:// URL. This comes in handy during network outages or congestion, and helps me build an overview of the protocol by organizing the documentation.

I also go and hunt down the FAQs and other information about the protocol on the Internet, since the writing style on RFCs often makes my head swim. Once I have a grasp on the protocol, I generally just hit the RFCs for the information I need, but it can take a while to assimilate all the details that I need to navigate the RFCs effectively.

Ethernet

A way to move lots of data around relatively short distances. Hubs work at the Ethernet level— they’re basically just smart wires that know how to keep the right voltages on a line. Check out the EtherApe project for a graphical network monitor for Unix, and the Ethereal Network Analyzer for a protocol analyzer. tcpdump is a handy utility based on libpcap. libnet is good for building up arbitrary packets.

ARP: Address Resolution Protocol

The way machines convert between Ethernet addresses and IP addresses.

IP: Internet Protocol

 The basic protocol that gets packets of data from one place to the other on the Internet. Routers and switches operate at this level, making sure packets get to their proper destinations.

TCP: Transmission Control Protocol

 A protocol that sits on top of IP and gives you the illusion that you have a reliable connection to another spot on the Internet. This is what you get when you ask for a SOCK_STREAM socket.

UDP: User Datagram Protocol

 A protocol that just ships packets around and lets you worry about the contents. This is what you get when you ask for a SOCK_DGRAM socket.

ICMP: Internet Control Message Protocol

 Short packets used for sending information about your connectivity. This is what goes on under the covers of utilities like ping.

PPP: Point to Point Protocol

 A way of doing IP networking over non-IP transports; the usual way to get a computer onto the Internet via a modem. There are a lot of RFCs dealing with PPP; I have not linked to all of them here.

RFCTitleUpdates or Obsoletes RFCs...
791 Internet Protocol (STD0005) 760
792 Internet Control Message Protocol (STD0005) 777, 760
768 User Datagram Protocol (STD0006)  
793 Transmission Control Protocol (STD0007)  
1661 The Point-to-Point Protocol (PPP) (STD0051) 1548, 1331, 1171, 1134
1332 The PPP Internet Protocol Control Protocol (IPCP) 1172

DNS: Domain Name Service

How to translate names like “www.amurgsval.org” into IP addresses like “209.24.112.118”. Used by programs like nslookup.

Telnet

 One of the most basic ways of getting out on the Net— basically a TCP connection with some extra option negotiation tucked in. There’s a complete option list (except for RFC1097: Telnet Subliminal-Message Option) at IANA.
RFCTitleUpdates or Obsoletes RFCs...
854 Telnet Protocol Specification (STD0008)  
855 Telnet Option Specifications (STD0008)  
856 Telnet Binary Transmission (STD0027)  
857 Telnet Echo Option (STD0028)  
858 Telnet Suppress Go Ahead Option (STD0029)  
859 Telnet Status Option (STD0030) 651
860 Telnet Timing Mark Option (STD0031)  
861 Telnet Extended Options - List Option (STD0032)  
885 Telnet End of Record Option  
1184 Telnet Linemode Option 1116

E-mail

 One of the oldest applications on the Internet. If you ever find yourself implementing an RFC822 E-mail parser, be warned: not all mailers are not fully RFC822 compliant and may give you occasional problems with fields like From:.

MIME: Multipurpose Internet Mail Extensions

 The format for shipping arbitrary chunks of data via E-mail. Also used in HTTP and other protocols as well.

SMTP: Simple Mail Transport Protocol

 The way messages get around the Internet.

POP3: Post Office Protocol

 A way to retrieve your mail online.

IMAP: Internet Message Access Protocol

 Another way to retrieve your mail online.

RFCTitleUpdates or Obsoletes RFCs...
822 Standard for the format of ARPA Internet text messages (STD0011) 733, 724
2045 MIME Part One: Format of Internet Message Bodies 1590, 1522, 1521, 1342, 1341
2046 MIME Part Two: Media Types. N. Freed, N. Borenstein
2047 MIME Part Three: Message Header Extensions for Non-ASCII Text.
2048 MIME Part Four: Registration Procedures (BCP0013)
2049 MIME Part Five: Conformance Criteria and Examples
2231 MIME Parameter Value and Encoded Word Extensions: Character Sets, Languages, and Continuations 2184, 2183, 2047, 2045, 1806
2646 The Text/Plain Format Parameter 2046
2854 The 'text/html' Media Type 2070, 1980, 1942, 1867, 1866, 1896, 1563, 1523
3023 XML Media Types 2376, 2048
821 Simple Mail Transfer Protocol (STD0010) 788, 780, 772
2821 Simple Mail Transfer Protocol 821, 974, 1689, 1123
2822 Internet Message Format 822
1939 Post Office Protocol - Version 3 (STD0053) 1725, 1460, 1225, 1081, 0937, 0918
1082 Post Office Protocol: Version 3: Extended service offerings  
1957 Some Observations on Implementations of the Post Office Protocol (POP3) 1939
2449 POP3 Extension Mechanism 1939
2060 Internet Message Access Protocol - Version 4rev1 1730
2061 IMAP4 Compatibility with IMAP2bis 1730

NNTP: Network News Transport Protocol

FTP: File Transfer Protocol

TFTP: Trivial File Transfer Protocol

HTTP

 Base64 and Digest auth, ... My own experience with Microsoft’s proprietary authentication...

HTTP Servers

Apache

A popular and free web server.

IIS

ISAPI...

Netscape

NSAPI...

Tux

A screamingly fast Linux-based web server that works at the kernel level.

WebDAV

WebDAV stands for “Web-based Distributed Authoring and Versioning”; in essence, it’s a way to turn HTTP into a file system.

Stress Testers

LoadSim

WAP: Wireless Application Protocol

My professional opinion is that the lifetime of WAP is going to run out fairly quickly: there’s no way that WAP devices with their hard-to-upgrade browsers are going to be able to evolve as fast as web browsers did back in the ’90’s. I expect that technology like AvantGo’s will leave WAP in the dust as cellphones and other WAP devices become more programmable.

DHCP: Dynamic Host Configuration Protocol

 FAQ at The DHCP Handbook. DHCP.org. The Internet Software Consortium’s DHCP Distribution is free and comes standard with Red Hat Linux.

SNMP: Simple Network Management Protocol

SNMP (check out SNMPLink and SNMPWORLD) is the most popular protocol for managing devices remotely (though take a look at the Distributed Management Task Force for a possible competitor); it’s supposed to be relatively lightweight and UDP-based, but is anything but simple these days. There are multiple versions of the protocol: v1 is the simplest and most common; v2c is the portion of the v2 specficiation that people found practical to implement; v3 is quite new, but actually has security. Check out the SNMP FAQ. The NET-SNMP Project has a library that handles v1, v2c, and v3 quite well. There’s a collection of MIBs at ISI. OpenNMS, GxSNMP, myNMS, and Nomad are open source entrants to the playing field of HP OpenView and Tivoli. libsmi, a library that allows management applications to access SMI MIB module definitions. Once you’ve got your SNMP agent working, visit InterWorking Labs, Inc., makers of SNMP test software, so you can make sure your implementation is solid. I’m only presenting a small fraction of the SNMP-related RFCs here (though a large chunk of those are MIBs).

RFCTitleUpdates or Obsoletes RFCs...
1157 Simple Network Management Protocol 1098, 1067
1905 Protocol Operations for Version 2 of the Simple Network Management Protocol (SNMPv2) 1448
1156 Management Information Base for Network Management of TCP/IP-based internets (MIB-I) 1066
1213 Management Information Base for Network Management of TCP/IP-based internets: MIB-II 1158
1155 Structure and Identification of Management Information for TCP/IP-based Internets (SMIv1) 1065
2578 Structure of Management Information Version 2 (SMIv2) (STD0058) 1902, 1442
2579 Textual Conventions for SMIv2 (STD0058) 1903, 1443
2576 Coexistence between Version 1, Version 2, and Version 3 of the Internet-standard Network Management Framework 2089, 1908, 1452

AAA: Authentication, Authorization, and Accounting

AAA servers are needed by ISPs to keep track of their thousands of users. Cisco has an AAA Case Study Overview.

LDAP: Lightweight Directory Access Protocol

 LDAP is used for storing user information, both for lookup purposes and for authentication. OpenLDAP is a free server.

RADIUS: Remote Authentication Dial In User Service

 RADIUS is the current IETF standard for AAA, though the AAA working group are working on the next generation, Diameter (see RFC2882 if you want to know why they need a next generation). Of interest: Bruce Morrison’s The RADIUS protocol and Applications. The FreeRADIUS Server Project has the Cistron server right now, and will have a fully functional free server later; they also host modules for PAM and Apache that will authenticate users against a RADIUS server. Commercial RADIUS solutions include:

TACACS+

 Cisco Systems favors the TACACS+ protocol, though many of their products now use RADIUS as well. Commercial solutions include:

RFCTitleUpdates or Obsoletes RFCs...
2251 Lightweight Directory Access Protocol v3  
2252 LDAP v3: Attribute Syntax Definitions  
2253 LDAP v3: UTF-8 String Representation of Distinguished Names  
2254 The String Representation of LDAP Search Filters  
2255 The LDAP URL Format  
2256 A Summary of the X.500(96) User Schema for use with LDAPv3  
2865 Remote Authentication Dial In User Service (RADIUS) 2138, 2058
2866 RADIUS Accounting 2139, 2059
2867 RADIUS Accounting Modifications for Tunnel Protocol Support 2866
2868 RADIUS Attributes for Tunnel Protocol Support 2865
2869 RADIUS Extensions  
2882 Network Access Servers Requirements: Extended RADIUS Practices  
2618 RADIUS Authentication Client MIB  
2619 RADIUS Authentication Server MIB  
2620 RADIUS Accounting Client MIB  
2621 RADIUS Accounting Server MIB  
2548 Microsoft Vendor-specific RADIUS Attributes  
TACACS+ The TACACS+ Protocol Version 1.78 TACACS+ 1.76

MPEG and MP3

MP3 is short for MPEG Audio Layer 3, which is a standard for compression from the Moving Picture Experts Group. Other MPEG standards are used in DVD’s and VCD’s. MP3 is a lossy form of audio compression, like JPEG is for images, and is usually good for about a megabyte per minute of music. Reading the bits off of a CD into your computer is called ripping, and it isn’t always easy— CD-ROMs follow a slightly different format than CDs, and it takes some decent software to compensate between the different data formats. Services like Napster, Gnutella (with clients like BearShare), and MusicCity allow you to swap MP3 files online. In my opinion, they aren’t exactly a threat to the recording industry— half the files I find on Napster are truncated before the file reaches the end, and the way Napster is coded makes it very easy for this to happen. I occasionally use Napster to discover new music, but that’s so I can go and buy the CD for myself, if it’s available. My primary use for MP3s is burning CDs that I can play in my car-based MP3 CD player— one CD can easily hold twelve hours of music.

SGML: Standard Generalized Markup Language

 A document markup language so complicated that people make wildly popular standards from its subsets.

HTML: HyperText Markup Language

 Netscape’s HTML Tag Reference is my usual reference point for HTML. Their JavaScript Reference is also very good, though Internet Explorer doesn’t always implement features present there. (If you’re using Mozilla’s JavaScript engine, the Netscape reference is quite reliable.)

CSS: Cascading Style Sheets

Style sheets have been a big disappointment. The implementation in popular browsers is rarely compliant with the actual specifications. Dynamic HTML in Netscape Communicator is a good start. You’ll also need the MSDN reference on DHTML, HTML, and CSS for dealing with all the vagaries of IE.

XML: eXtensible Markup Language

 Okay, I’ve gotta say it: someone saying that their product is better because it involves XML is like having them say it’s better because it involves ASCII. XML itself is just another way to store data. You need to have one of the many formats like XSL, XQL, and so on in order for it to be significant... and everyone needs to implement the standard for XSL, XQL, etc. properly or you’ll just wind up with another mess like the way Cascading Style Sheets are now.

WML: Wireless Markup Language

 An XML variant for WAP.

Unicode

UTF-8,... The Unicode Test Page

Encryption and Security

Netscape Introduction to Public-Key Cryptography.

PGP: Pretty Good Privacy

A popular form of public key cryptography.

SSL: Secure Sockets Layer and TLS: Transport Layer Security

Netscape Introduction to SSL. OpenSSL toolkit.

IPSec: IP Security Protocol

Linux FreeS/WAN. Virtual Private Network Consortium.

SSH

A very handy protocol that provides secure remote logins, file transfer, and TCP/IP forwarding (including for X11). The OpenSSH project have good open source tools. SSH Communications Security have commercial tools.

Kerberos

A secure network authentication service. I haven’t had a chance to work with it yet.

RFCTitleUpdates or Obsoletes RFCs...
2246 The TLS Protocol Version 1.0  
2817 Upgrading to TLS Within HTTP/1.1 2616
2712 Addition of Kerberos Cipher Suites to Transport Layer Security (TLS)  
2818 HTTP Over TLS  
2411 IPSec Document Roadmap  
2401 Security Architecture for the Internet Protocol  
2402 IP Authentication Header  
2403 The Use of HMAC-MD5-96 within ESP and AH  
2404 The Use of HMAC-SHA-1-96 within ESP and AH  
2405 The ESP DES-CBC Cipher Algorithm With Explicit IV  
2406 IP Encapsulating Security Payload (ESP)  
2407 The Internet IP Security Domain of Interpretation for ISAKMP  
2408 Internet Security Association and Key Management Protocol (ISAKMP)  
2409 The Internet Key Exchange (IKE)  
2451 The ESP CBC-Mode Cipher Algorithms  

Copyright © 2001–4 Max Rible Kaehn — All Rights Reserved.