Categories: Linux

iptables – table full and dropping packets

nf_conntrack: table full, dropping packet


I actually saw this error in a Docker host, and Docker uses iptables and allof Docker’s iptables rules are added to the DOCKER chain. In this case though, it wasn’t the Docker iptables rules that were a problem, it is just that limits were reached in the connection tracking of the netfilter module. You might see this error in /var/log/messages or /var/log/kern

The full error message looked like this –

May 29 09:10:37 docker kernel: [74350.150400] nf_conntrack: table full, dropping packet
May 29 09:10:37 docker kernel: [74350.155361] nf_conntrack: table full, dropping packet
May 29 09:10:37 docker kernel: [74350.160282] nf_conntrack: table full, dropping packet
May 29 09:10:37 docker kernel: [74350.181547] nf_conntrack: table full, dropping packet
May 29 09:10:37 docker kernel: [74350.184807] nf_conntrack: table full, dropping packet
May 29 09:10:37 docker kernel: [74350.184913] nf_conntrack: table full, dropping packet

Resolution – increase maximum number of connections being tracked and/or reduce tracking timeouts. Look for these run time kernel parameters –

[root@kauai /]# sysctl net.ipv4.netfilter.ip_conntrack_tcp_timeout_established
net.ipv4.netfilter.ip_conntrack_tcp_timeout_established = 27000
[root@kauai /]# sysctl net.netfilter.nf_conntrack_generic_timeout
net.netfilter.nf_conntrack_generic_timeout = 60
[root@kauai /]# sysctl net.ipv4.netfilter.ip_conntrack_max
net.ipv4.netfilter.ip_conntrack_max = 64268

These are the settings which resolved my issue, simply doubled the values –

sysctl -w net.ipv4.netfilter.ip_conntrack_tcp_timeout_established=54000
sysctl -w net.netfilter.nf_conntrack_generic_timeout=120
sysctl -w net.ipv4.netfilter.ip_conntrack_max=128536

To make this permanent, add the lines above to the /etc/sysctl.conf file.

 

References –

https://security.stackexchange.com/questions/43205/nf-conntrack-table-full-dropping-packet

https://docs.docker.com/network/iptables/

daniel

Share
Published by
daniel

Recent Posts

GCP for Linux System administrators

Linux System Admins Journey to Google Cloud Platform As a Linux system administrator, you have…

9 months ago

Top 5 Troubleshooting Tools for Network Professionals in Linux

As a network professional, troubleshooting is a crucial part of your daily routine. To streamline…

9 months ago

netstat equivalent tool

The net-tools set of packages had been deprecated years back, although the commands are still…

2 years ago

GCP GKE – run kubectl through bastion host

Re-posting my answer to a Google cloud platform's Google Kubernetes Engine (GKE) related question in…

4 years ago

Spoof User Agent in http calls

Recently I was trying to download numerous files from a certain website using a shell…

4 years ago

Terraform – show logging

Enabling logging in terraform for debugging

5 years ago