How to setup IPSec interoperable for Linux, OpenBSD and PGPNet

Copyright 2000, Hans-Jörg Höxer
This document is still under construction and the information is partly untested and might be completely wrong.
Consider yourself warned.

Preface

This HOWTO is Copyrighted 2000 by Hans-Jörg Höxer. It can be distributed freely. It cannot be modified. If you have any kind of sugestion, please send me an email (I will update the document if the sugestion proceeds). No liability for the contents of this document can be accepted. I have no responsability about the consequences of following the steps provided in this document. If you have questions, please contact me at hshoexer@rommelwood.de.

Content

  • Introduction
  • Interop GNU/Linux and OpenBSD
  • Interop GNU/Linux and PGPNet
  • Interop OpenBSD and PGPNet
  • Summary

    • Introduction

    Aim of this document is to give some examples for setting up IPSec between different platforms. Tested operatingsystems were GNU/Linux using FreeS/WAN 1.5, OpenBSD 2.6 and Windows 98 using PGPFreeware 6.5.3. Given configfiles will be as minimal as possible focusing on an interoperable setup. Therefor a very simple LAN of severel boxes was used, so only host-to-host connections will be covered. But except for PGPNet, extending the following examples to reallife VPN-configurations will not affect the basic setup concerning interoperability. The free version of PGPNet only supports host-to-host connections.

    Interop GNU/Linux and OpenBSD

    The two boxes used are GNU with ip 192.168.2.1 and OBSD with ip 192.168.2.5. FreeS/WAN supports preshared secrets and RSA signatures for authentication. For older versions of Pluto exists a patch by Neil Dunbar to enable X-509 support. OpenBSD features also preshared secrets and X-509 certificates, but RSA signatures are not yet implented or at least, the documentation is missing. Therefor, in the example authentication is done using preshared secrets. The file /etc/ipsec.conf on GNU looks like this: 
       config setup
	interfaces="ipsec0=eth0"
	klipsdebug=none
	plutodebug=none
	plutoload=%search
	plutostart=%search
	plutowait=no

   conn GNU-OBSD
	auto=add
        type=tunnel
        
	left=192.168.2.1
        right=192.168.2.5
	
	keyexchange=ike
        keylife=8h
	keyingtries=5
	pfs=yes
	rekeymargin=9m
	rekeyfuzz=25%
    In /etc/ipsec.secrets the secret passphrase is specified: 
        192.168.2.1 192.168.2.5: PSK "this_is_our_badly_chosen_secret_passphrase"
    This is /etc/isakmpd/isakmpd.conf on OBSD: 
        [General]
    Retransmits=		5
    Exchange-max-time=	120
    Listen-on=		192.168.2.5
    Shared-SADB=		Defined
    
    
    # Incoming phase 1 negotiations are multiplexed on the source IP address
    [Phase 1]
    192.168.2.1=		ISAKMP-peer-GNU
    Default=		ISAKMP-peer-GNU
    
    
    
    # These connections are walked over after config file parsing and told
    # to the application layer so that it will inform us when traffic wants to
    # pass over them.  This means we can do on-demand keying.
    [Phase 2]
    Connections=		IPsec-OBSD-GNU
    
    
    
    # The peers
    [ISAKMP-peer-GNU]
    Phase=		1
    Transport=	udp
    Local-address=	192.168.2.5
    Address=	192.168.2.1
    Configuration=	Default-main-mode
    Authentication=	this_is_our_badly_chosen_secret_passphrase
    
    [ISAKMP-peer-GNU-aggressive]
    Phase=		1
    Transport=	udp
    Local-address=	192.168.2.5
    Address=	192.168.2.1
    Configuration=	Default-aggressive-mode
    Authentication=	this_is_our_badly_chosen_secret_passphrase
    
    
    
    
    # The different connections
    [IPsec-OBSD-GNU]
    Phase=		2
    ISAKMP-peer=	ISAKMP-peer-GNU
    Configuration=	Default-quick-mode
    Local-ID=	Net-OBSD
    Remote-ID=	Net-GNU
    
    
    # Certificates stored in PEM format
    [X509-certificates]
    CA-directory=           /etc/isakmpd/ca/
    Cert-directory=         /etc/isakmpd/certs/
    Private-key=            /etc/isakmpd/private/local.key
    
    
    # Our Networks
    [Net-GNU]
    ID-type=	IPV4_ADDR_SUBNET
    Network=	192.168.2.1
    Netmask=	255.255.255.255
    
    [Net-OBSD]
    ID-type=	IPV4_ADDR_SUBNET
    Network=	192.168.2.5
    Netmask=	255.255.255.255
    
    
    # Phase 1 descriptions
    [Default-main-mode]
    DOI=		IPSEC
    EXCHANGE_TYPE=	ID_PROT
    Transforms=	3DES-SHA,3DES-MD5
    
    [Default-aggressive-mode]
    DOI=		IPSEC
    EXCHANGE_TYPE=	AGGRESSIVE
    Transforms=	3DES-SHA,3DES-MD5
    
    
    # Main mode transforms
    ########################
    # 3DES
    
    [3DES-SHA]
    ENCRYPTION_ALGORITHM=	3DES_CBC
    HASH_ALGORITHM=		SHA
    AUTHENTICATION_METHOD=	PRE_SHARED
    GROUP_DESCRIPTION=	MODP_1024
    Life=			LIFE_180_SECS
    
    [3DES-MD5]
    ENCRYPTION_ALGORITHM=   3DES_CBC
    HASH_ALGORITHM=         MD5
    AUTHENTICATION_METHOD=  PRE_SHARED
    GROUP_DESCRIPTION=      MODP_1024
    Life=                   LIFE_180_SECS
    
    
    # Quick mode description
    ########################
    
    [Default-quick-mode]
    DOI=			IPSEC
    EXCHANGE_TYPE=		QUICK_MODE
    Suites=			QM-ESP-3DES-SHA-PFS-SUITE,QM-ESP-3DES-MD5-PFS-SUITE
    
    
    # Quick mode protection suites
    ##############################
    # 3DES
    
    [QM-ESP-3DES-SHA-PFS-SUITE]
    Protocols=		QM-ESP-3DES-SHA-PFS
    
    [QM-ESP-3DES-MD5-PFS-SUITE]
    Protocols=		QM-ESP-3DES-MD5-PFS
    
    [QM-ESP-3DES-SHA-SUITE]	
    Protocols=		QM-ESP-3DES-SHA
    
    [QM-ESP-3DES-MD5-SUITE]
    Protocols=		QM-ESP-3DES-MD5
    
    
    # Quick mode protocols
    #############################    
    # 3DES
    
    [QM-ESP-3DES-SHA-PFS]
    PROTOCOL_ID=	        IPSEC_ESP
    Transforms=		QM-ESP-3DES-SHA-PFS-XF
    
    [QM-ESP-3DES-SHA]
    PROTOCOL_ID=		IPSEC_ESP
    Transforms=		QM-ESP-3DES-SHA-XF
    
    [QM-ESP-3DES-MD5-PFS]
    PROTOCOL_ID=		IPSEC_ESP
    Transforms=		QM-ESP-3DES-MD5-PFS-XF
    
    [QM-ESP-3DES-MD5]
    PROTOCOL_ID=	        IPSEC_ESP
    Transforms=		QM-ESP-3DES-MD5-XF
    
    
    
    # Quick mode transforms
    #############################    
    # 3DES
    
    [QM-ESP-3DES-SHA-PFS-XF]
    TRANSFORM_ID=			3DES
    ENCAPSULATION_MODE=		TUNNEL
    AUTHENTICATION_ALGORITHM=	HMAC_SHA
    GROUP_DESCRIPTION=	        MODP_1024
    Life=				LIFE_3600_SECS
    
    [QM-ESP-3DES-SHA-XF]
    TRANSFORM_ID=			3DES
    ENCAPSULATION_MODE=		TUNNEL
    AUTHENTICATION_ALGORITHM=	HMAC_SHA
    GROUP_DESCRIPTION=		MODP_1024
    Life=				LIFE_3600_SECS
    
    [QM-ESP-3DES-MD5-PFS-XF]
    TRANSFORM_ID=			3DES
    ENCAPSULATION_MODE=		TUNNEL
    AUTHENTICATION_ALGORITHM=	HMAC_MD5
    GROUP_DESCRIPTION=		MODP_1024
    Life=				LIFE_3600_SECS
    
    [QM-ESP-3DES-MD5-XF]
    TRANSFORM_ID=			3DES
    ENCAPSULATION_MODE=		TUNNEL
    AUTHENTICAION_ALGORITHM=	HMAC_MD5
    GROUP_DESCRIPTION=		MODP_1024
    Life=				LIFE_3600_SECS
    
    
    
    [LIFE_8_HOURS]
    LIFE_TYPE=		SECONDS
    LIFE_DURATION=		28800,25200:32400
    
    [LIFE_1_DAY]
    LIFE_TYPE=		SECONDS
    LIFE_DURATION=		86400,79200:93600
    
    [LIFE_180_SECS]
    LIFE_TYPE=		SECONDS
    LIFE_DURATION=		180,120:240
    
    [LIFE_3600_SECS]
    LIFE_TYPE=		SECONDS
    LIFE_DURATION=		3600,1800:7200
    OpenBSDs isakmpd is highly configurable, features DES, 3DES, Blowfish and CAST encryption algorithms, different keylifetimes based on time or transfered data. As FreeS/WAN only supports 3DES, MD5 and SHA the isakmpd.conf was cut down to fit these needs.

    This is the corresponding /etc/isakmpd.policy on OBSD: 

        KeyNote-Version: 2
    Comment: This policy accepts ESP SAs from a remote that uses the right password
    Authorizer: "POLICY"
    Licensees: "passphrase:this_is_our_badly_chosen_secret_passphrase"
    Conditions: app_domain == "IPsec policy" &&
	        esp_present == "yes" -> "true";

    Interop GNU/Linux and PGPNet

    To make this work, PGPNet was told to use only 3DES for encryption and SHA or MD5 for authentication. Further the keylifetime was reduced to 3600 seconds as used by FreeS/WAN. The GNU/Linux box is again GNU with ip 192.168.2.1 and for the Windows machine the name is WIN and the ip is 192.168.2.10. The conn-section of GNUs /etc/ipsec.conf is quite straight forward: 
        config setup
	interfaces="ipsec0=eth0"
	klipsdebug=none
	plutodebug=none
	plutoload=%search
	plutostart=%search
	plutowait=no


    conn GNU-WIN
	auto=add
	type=tunnel
	
	left=192.168.2.1
	right=192.168.2.10
	keyexchange=ike
	keylife=8h
	keyingtries=3
	pfs=yes
	rekeymargin=9m
	rekeyfuzz=25%
    Note that the tag auto=add is used instead of auto=start. The reason is that PGPNet expects to start the connection manually. If auto=start would be specified the Pluto demon would try to negotiate a security association but PGPNet would not answer. But initiating the connection by PGPNet still works, of course.

    As the public/secret key format used by PGPNet is not compatible with FreeS/WANs shared secrets must be used again. So /etc/ispec.secrets on GNUs looks like this: 

        192.168.2.1 192.168.2.10: PSK "this_is_another_badly_chosen_passphrase"
    On WIN the available algorithms and hashes were reduced to 3DES, MD5 and SHA (main menu -> options):

    The keylifetime for IKE was set to 3600 seconds and to 8 hours for IPSec:

    Then a new host was added...

    ... and encryption enforced...

    ...a descriptive name choosen...

    ...GNUs ip was setted...

    ...authentication by shared secret was choosen...

    ...identification by ip...

    ...and everything's ok:

    Remark: During the writing of this document a patch by Kai Martius for RSA based authentication was released. So a description for FreeS/WAN and PGPNet with RSA authentication will follow in a newer version of this howto.

    Interop OpenBSD and PGPNet

    As isakmpd is highly configureable it is quite easy to describe the transforms, protections suites and protocols suiting those provided by PGPNet. To keep things simple, the following example for /etc/isakmpd/isakmpd.conf on OBSD uses only these three transforms: CAST-SHA-1536, CAST-MD5-1024 and 3DES-MD5-1024 (cipher-hash-DHgroup). Again, for phase two the Passive-connection tag was used, as PGPNet expects to initiate the connection. The OpenBSD host is OBSD with ip 192.168.2.5 and the winbox is again WIN with ip 192.168.2.10. 
        [General]
    Retransmits=		5
    Exchange-max-time=	120
    Listen-on=		192.168.2.5
    Shared-SADB=		Defined
    
    
    # Incoming phase 1 negotiations are multiplexed on the source IP address
    [Phase 1]
    192.168.2.10=		ISAKMP-peer-WIN
    Default=		ISAKMP-peer-WIN
    
    
    # These connections are walked over after config file parsing and told
    # to the application layer so that it will inform us when traffic wants to
    # pass over them.  This means we can do on-demand keying.
    [Phase 2]
    Passive-connections=	IPsec-OBSD-WIN
    
    
    
    # The peers
    [ISAKMP-peer-WIN]
    Phase=		1
    Transport=	udp
    Local-address=	192.168.2.5
    Address=	192.168.2.10
    Configuration=	Default-main-mode
    Authentication=	this_is_another_badly_chosen_passphrase
    
    
    
    # The different connections
    [IPsec-OBSD-WIN]
    Phase=		2
    ISAKMP-peer=	ISAKMP-peer-WIN
    Configuration=	Default-quick-mode
    Local-ID=	Net-OBSD
    Remote-ID=	Net-WIN
    
    
    # Certificates stored in PEM format
    [X509-certificates]
    CA-directory=           /etc/isakmpd/ca/
    Cert-directory=         /etc/isakmpd/certs/
    Private-key=            /etc/isakmpd/private/local.key
    
    
    # Our Networks
    [Net-OBSD]
    ID-type=	IPV4_ADDR_SUBNET
    Network=	192.168.2.5
    Netmask=	255.255.255.255
    
    [Net-WIN]
    ID-type=	IPV4_ADDR_SUBNET
    Network=	192.168.2.10
    Netmask=	255.255.255.255
    
    
    
    # Phase 1 descriptions
    [Default-main-mode]
    DOI=		IPSEC
    EXCHANGE_TYPE=	ID_PROT
    Transforms=	3DES-SHA,3DES-MD5
    
    [Default-aggressive-mode]
    DOI=		IPSEC
    EXCHANGE_TYPE=	AGGRESSIVE
    Transforms=	3DES-SHA,3DES-MD5
    
    
    # Main mode transforms
    ########################
    # 3DES
    
    [3DES-SHA]
    ENCRYPTION_ALGORITHM=	3DES_CBC
    HASH_ALGORITHM=		SHA
    AUTHENTICATION_METHOD=	PRE_SHARED
    GROUP_DESCRIPTION=	MODP_1024
    Life=			LIFE_180_SECS
    
    [3DES-MD5]
    ENCRYPTION_ALGORITHM=   3DES_CBC
    HASH_ALGORITHM=         MD5
    AUTHENTICATION_METHOD=  PRE_SHARED
    GROUP_DESCRIPTION=      MODP_1024
    Life=                   LIFE_180_SECS
    
    
    # CAST
    
    [CAST-SHA-1536]
    ENCRYPTION_ALGORITHM=	CAST_CBC
    HASH_ALGORITHM=		SHA
    AUTHENTICATION_METHOD=	PRE_SHARED
    GROUP_DESCRIPTION=	MODP_1536
    Life=			LIFE_1_DAY
    
    [CAST-MD5-1024]
    ENCRYPTION_ALGORITHM=	CAST_CBC
    HASH_ALGORITHM=		MD5
    AUTHENTICATION_METHOD=	PRE_SHARED
    GROUP_DESCRIPTION=	MODP_1024
    Life=			LIFE_1_DAY
    
    
    # Quick mode description
    ########################
    
    [Default-quick-mode]
    DOI=			IPSEC
    EXCHANGE_TYPE=		QUICK_MODE
    Suites=			QM-ESP-3DES-SHA-PFS-SUITE,QM-ESP-3DES-MD5-PFS-SUITE
    
    
    # Quick mode protection suites
    ##############################
    # 3DES
    
    [QM-ESP-3DES-SHA-PFS-SUITE]
    Protocols=		QM-ESP-3DES-SHA-PFS
    
    [QM-ESP-3DES-MD5-PFS-SUITE]
    Protocols=		QM-ESP-3DES-MD5-PFS
    
    [QM-ESP-3DES-SHA-SUITE]	
    Protocols=		QM-ESP-3DES-SHA
    
    [QM-ESP-3DES-MD5-SUITE]
    Protocols=		QM-ESP-3DES-MD5
    
    # CAST
    
    [QM-ESP-CAST-SHA-SUITE]
    Protocols=		QM-ESP-CAST-SHA
    
    [QM-ESP-CAST-MD5-SUITE]
    Protocols=		QM-ESP-CAST-MD5
    
    
    # Quick mode protocols
    #############################    
    # 3DES
    
    [QM-ESP-3DES-SHA-PFS]
    PROTOCOL_ID=	        IPSEC_ESP
    Transforms=		QM-ESP-3DES-SHA-PFS-XF
    
    [QM-ESP-3DES-SHA]
    PROTOCOL_ID=		IPSEC_ESP
    Transforms=		QM-ESP-3DES-SHA-XF
    
    [QM-ESP-3DES-MD5-PFS]
    PROTOCOL_ID=		IPSEC_ESP
    Transforms=		QM-ESP-3DES-MD5-PFS-XF
    
    [QM-ESP-3DES-MD5]
    PROTOCOL_ID=	        IPSEC_ESP
    Transforms=		QM-ESP-3DES-MD5-XF
    
    
    # CAST
    
    [QM-ESP-CAST-SHA]
    PROTOCOL_ID=		IPSEC_ESP
    Transforms=		QM-ESP-CAST-SHA-XF
    
    [QM-ESP-CAST-MD5]
    PROTOCOL_ID=		IPSEC_ESP
    Transforms=		QM-ESP-CAST-MD5-XF
    
    
    # Quick mode transforms
    #############################    
    # 3DES
    
    [QM-ESP-3DES-SHA-PFS-XF]
    TRANSFORM_ID=			3DES
    ENCAPSULATION_MODE=		TUNNEL
    AUTHENTICATION_ALGORITHM=	HMAC_SHA
    GROUP_DESCRIPTION=	        MODP_1024
    Life=				LIFE_3600_SECS
    
    [QM-ESP-3DES-SHA-XF]
    TRANSFORM_ID=			3DES
    ENCAPSULATION_MODE=		TUNNEL
    AUTHENTICATION_ALGORITHM=	HMAC_SHA
    GROUP_DESCRIPTION=		MODP_1024
    Life=				LIFE_3600_SECS
    
    [QM-ESP-3DES-MD5-PFS-XF]
    TRANSFORM_ID=			3DES
    ENCAPSULATION_MODE=		TUNNEL
    AUTHENTICATION_ALGORITHM=	HMAC_MD5
    GROUP_DESCRIPTION=		MODP_1024
    Life=				LIFE_3600_SECS
    
    [QM-ESP-3DES-MD5-XF]
    TRANSFORM_ID=			3DES
    ENCAPSULATION_MODE=		TUNNEL
    AUTHENTICAION_ALGORITHM=	HMAC_MD5
    GROUP_DESCRIPTION=		MODP_1024
    Life=				LIFE_3600_SECS
    
    
    # CAST
    
    [QM-ESP-CAST-SHA-XF]
    TRANSFORM_ID=			CAST
    ENCAPSULATION_MODE=		TUNNEL
    GROUP_DESCRIPTION=		MODP_1536
    AUTHENTICATION_ALGORITHM=	HMAC_SHA
    Life=				LIFE_8_HOURS
    
    [QM-ESP-CAST-MD5-XF]
    TRANSFORM_ID=			CAST
    ENCAPSULATION_MODE=		TUNNEL
    GROUP_DESCRIPTION=		MODP_1024
    AUTHENTICATION_ALGORITHM=	HMAC_MD5
    Life=				LIFE_8_HOURS
    
    [LIFE_8_HOURS]
    LIFE_TYPE=		SECONDS
    LIFE_DURATION=		28800,25200:32400
    
    [LIFE_1_DAY]
    LIFE_TYPE=		SECONDS
    LIFE_DURATION=		86400,79200:93600
    
    [LIFE_180_SECS]
    LIFE_TYPE=		SECONDS
    LIFE_DURATION=		180,120:240
    
    [LIFE_3600_SECS]
    LIFE_TYPE=		SECONDS
    LIFE_DURATION=		3600,1800:7200
    Here is the corresponing /etc/isakmpd.policy: 
        KeyNote-Version: 2
    Comment: This policy accepts ESP SAs from a remote that uses the right password
    Authorizer: "POLICY"
    Licensees: "this_is_another_badly_chosen_passphrase"
    Conditions: app_domain == "IPsec policy" &&
	        esp_present == "yes" -> "true";
    On WIN things were quite straight forward as the default algorithms were used and the keylifetimes were again set to 3600 seconds for IKE and 8 hours for IPSec. Then a new host was added the same way as in the previous section (see above for details).

    Summary

    Finally, putting all together we get these config files:

  • /etc/ipsec.conf

  • /etc/ipsec.secrets

  • /etc/isakmpd/isakmpd.conf

  • /etc/isakmpd.policy

    • For PGPNet just configure two host, one for OpenBSD and one for GNU, as described above.

    Enjoy!