Cdtjkyj

Currently I'm trying to troubleshoot an e-mail server that will allow users to send e-mail, but not log in to the server (POP3) or receive e-mail using their Outlook clients. Previously this was doable, but moving the server to it's current location broke it. For obvious reasons, I suspect it's the firewall causing issues.

Firstly, here is what the handshake (or attempt at) looked like before I made any changes to the firewall:

• 
  mail-server: 192.168.25.26


• 
  

  client: 192.168.25.50

  
  
  

       Source     |     Destination     |   PROTO  |     INFO
  
  192.168.25.50        192.168.25.26         TCP        50861 > pop3 [SYN] seq=0 win=65535 len=0
  
  192.168.25.26        192.168.25.50         TCP        pop3 > 58601 [RST, ACK] seq=1 win=1 len=0
  
  192.168.25.50        192.168.25.26         TCP        [TCP Retransmission] 58061 > pop3 [Syn] etc....
  
  

  
  

and the [rst, ack], [tcp re-transmission] dance happens once more before failing.

The rules for the firewall (Cisco) concerning 192.168.25.26 and POP3 are as follows:

   source     |     destination     |        service     

     any           192.168.25.0/24          tcp/pop3

192.168.25.0/24          any                tcp/smtp

                  208.105.121.196

   source     |     destination     |        service     

     any           192.168.25.26            tcp/pop3

                        any                 tcp/smtp

To me, that looks fine. To be certain, on the email server (Zentyal) I ran the following commands:

(IP tables was HEAVILY configured. I suspect the issue lies here. Is there any way to allow all traffic, just for testing's sake?)

sudo iptables -F

sudo iptables -P INPUT ACCEPT

sudo iptables -P FORWARD ACCEPT

sudo iptables -P OUTPUT ACCEPT

However, there's still no outlook connect-ability. At this point, on the email server I ran the command:

telnet 127.0.0.1 110

and was able to log in to the mail server. From a client inside the network, I ran the same command and was told the "connection failed".

After these changes, when I attempt to "test the settings" in Outlook, here's what the handshake looks like. I noticed something peculiar.

     Source     |     Destination     |   PROTO  |     INFO

192.168.25.50        192.168.25.26         TCP        apollo-status > pop3 [SYN] seq=0 win=65535 len=0

192.168.25.26        192.168.25.50         TCP        pop3 > apollo-status [RST, ACK] seq=1 win=1 len=0

192.168.25.50        192.168.25.26         TCP        [TCP Retransmission] apollo-status > pop3 [Syn] etc....

and it does it twice more, like the last time. In the INFO pane, I now see apollo-status instead of a port number. The next two times I ran it, I saw npep_messaging and synapse, respectively. I suspect these are random ports chosen by dovecot, because in dovecot.config I changed the default listener to an asterisk "*" from the port previously specified, which seemed to be an arbitrary port in the 4,000's and had no rules in the firewall.

Just to confirm, it seems the port it uses each time is changing based on the (recommended) configuration. On the loopback interface port 110 clearly has a listener attached to it, but from another computer, it appears to be closed.

Any help at all is appreciated. I've racked my brain with this, and I'm hoping that this community will have some novel ideas.

so the RST message shouldn't freak you out. It occurs whenever you attempt to access a service that is firewalled or the OS is telling you that there's no such service listening on this port (Incidentally, this trick is used by traceroute to find out when traceroute needs to stop it's probing.).

Let's take a look at what I am TCP sockets are in the Listen state on my box

❯ netstat -an | grep LISTEN                                      [11:49:58 PM]

tcp4       0      0  *.9100                 *.*                    LISTEN

tcp4       0      0  *.17500                *.*                    LISTEN

tcp4       0      0  127.0.0.1.8021         *.*                    LISTEN

tcp6       0      0  ::1.8021               *.*                    LISTEN

tcp4       0      0  127.0.0.1.3306         *.*                    LISTEN

tcp4       0      0  *.22                   *.*                    LISTEN

tcp6       0      0  *.22                   *.*                    LISTEN

tcp4       0      0  127.0.0.1.631          *.*                    LISTEN

tcp6       0      0  ::1.631                *.*                    LISTEN

So let's try to connect to a non existent service and at the same time snoop on my lo traffic.

 ❯ nc -v -z 127.0.0.1 12002                                       [11:50:12 PM]

nc: connectx to 127.0.0.1 port 12002 (tcp) failed: Connection refused

Here's the OS telling me that there's no such service here. Firewalls fake it the exact same way (the nice ones when you use the REJECT destination. Destination DROP just /dev/null your traffic).

❯ sudo tshark -i lo -s0                                                                                                                                [11:50:04 PM]

Capturing on 'Loopback'

  1   0.000000    127.0.0.1 -> 127.0.0.1    TCP 68 54188→12002 [SYN] Seq=0 Win=65535 Len=0 MSS=16344 WS=32 TSval=688032360 TSecr=0 SACK_PERM=1

  2   0.000063    127.0.0.1 -> 127.0.0.1    TCP 44 12002→54188 [RST, ACK] Seq=1 Ack=1 Win=0 Len=0

Connecting to an active port results in the TCP handshake & teardown dance

 ❯ nc -v -z 127.0.0.1 8021                                        [11:50:23 PM]

found 0 associations

found 1 connections:

     1: flags=82<CONNECTED,PREFERRED>

    outif lo0

    src 127.0.0.1 port 54190

    dst 127.0.0.1 port 8021

    rank info not available

    TCP aux info available

And the relevant network traffic:

  3  11.123990    127.0.0.1 -> 127.0.0.1    TCP 68 54190→8021 [SYN] Seq=0 Win=65535 Len=0 MSS=16344 WS=32 TSval=688043453 TSecr=0 SACK_PERM=1

  4  11.124082    127.0.0.1 -> 127.0.0.1    TCP 68 8021→54190 [SYN, ACK] Seq=0 Ack=1 Win=65535 Len=0 MSS=16344 WS=32 TSval=688043453 TSecr=688043453 SACK_PERM=1

  5  11.124091    127.0.0.1 -> 127.0.0.1    TCP 56 54190→8021 [ACK] Seq=1 Ack=1 Win=408288 Len=0 TSval=688043453 TSecr=688043453

  6  11.124099    127.0.0.1 -> 127.0.0.1    TCP 56 [TCP Window Update] 8021→54190 [ACK] Seq=1 Ack=1 Win=408288 Len=0 TSval=688043453 TSecr=688043453

  7  11.136377    127.0.0.1 -> 127.0.0.1    TCP 56 54190→8021 [FIN, ACK] Seq=1 Ack=1 Win=408288 Len=0 TSval=688043464 TSecr=688043453

  8  11.136406    127.0.0.1 -> 127.0.0.1    TCP 56 8021→54190 [ACK] Seq=1 Ack=2 Win=408288 Len=0 TSval=688043464 TSecr=688043464

  9  11.136415    127.0.0.1 -> 127.0.0.1    TCP 56 [TCP Dup ACK 7#1] 54190→8021 [ACK] Seq=2 Ack=1 Win=408288 Len=0 TSval=688043464 TSecr=688043464

 10  11.263097    127.0.0.1 -> 127.0.0.1    TCP 56 8021→54190 [FIN, ACK] Seq=1 Ack=2 Win=408288 Len=0 TSval=688043587 TSecr=688043464

 11  11.263126    127.0.0.1 -> 127.0.0.1    TCP 56 54190→8021 [ACK] Seq=2 Ack=2 Win=408288 Len=0 TSval=688043587 TSecr=688043587

So how do you go about solving your issue?

Ensure that your socket is listening on an externally visible port (*:port_number)

❯ netstat -an | grep LISTEN                                      [11:49:58 PM]

tcp4       0      0  *.9100                 *.*                    LISTEN <<< COOL

tcp4       0      0  *.17500                *.*                    LISTEN

tcp4       0      0  127.0.0.1.8021         *.*                    LISTEN <<< NOT COOL

Start with the simplest firewall or no firewall and connect to the service port from a remote box on the same LAN. Remember to turn on the iptables forwarding kernel flag via sysctl
On the server end

sudo iptables  -n -L -X -v

sudo tshark -i eth0 -s0 port 110

on the client end

sudo tshark -i eth0 -s0 port 110

nc  -v -z mailserver.example.org 110

It's important to compare the tshark output from both ends to ensure that what you send is what gets to the remote box