Saturday, October 12, 2013

Flaw in common network design pattern (or in understanding how to use it)?

There is an old and not-so-clever network design pattern like this:



I know this from CCN(A|P) but I have no idea who invented it. Cisco perhaps. Idea is to use IGP, nowadays it is naturally OSPF in most cases, for resolution of the L3 path towards a subnet in question. The subnet itself is terminated on a gateway. Two different gateways to be accurate. One acts as a default gateway, which means that it is active router in HSRP, so it holds "virtual" IP address as well as MAC address of the default gateway. It is the address that hosts in the subnet uses in their routing tables and subsequently in their ARP tables.

So far so good. Everything is up and running, one gateway is active, another one passive, both routers are redistributing connected routes to OSPF and both are able to deliver traffic from L3 networks towards clients in the L2 subnet in question.

But wait a moment... How do L3 switches know where to send the traffic over L2? Yes, routing table and ARP resolution. By default the ARP table records timeout is 4 hrs, right?

And how do L2 switches know the port to forward traffic? MAC address table (or switching table, depends on terminology). Timeout for MAC address table? 5 mins.

So what does the passive switch do when no traffic from hosts hits this switch, because it does not have active default GW address and nobody wants to communicate over L2 to it's "real IP". Well, obviously the MAC address table cleans out after a while but ARP records stays much much longer (and ARP records are refreshed when some traffic arrives and the L3 switch does not have proper ARP table records).

But... When there is no MAC address table record for the particular MAC on the standby L3 switch, the switch just broadcasts the traffic for that MAC to all ports. And it multiplies in the network further because in case you have prevailing vertical flows in your network it is likely that any other L2 switch in the chain does not know the MAC address either. Except the one which has the destination host connected. So it means you might have huge multiplication of useless traffic if your L3 switch attracts some traffic in OSPF for any subnet which it is standby for it's gateway.

It might several megs or even tens or hundred megs of constant scattered traffic in real life scenario in 1 Gig network.

And what to do to avoid this? Think of the design pattern... Set higher OSPF costs for redistributed routes on the standby router. Simple but effective, huh?

Thursday, May 30, 2013

USB device ID 0bdb:1926 Ericsson Business Mobile Networks BV stopped working after update Linux Mint 14 to 15

I tried to upgrade Linux Mint on my laptop from Linux Mint 14 (Nadia) to 15 (Olivia). And suddenly GSM/3G modem that I am using to connect to one Czech mobile ISP stopped working.

The problem was that the connecting to the network timed out each time. And there was no apparent reason in /var/log/syslog apart from that DHCP did not received a reply from the ISP:

May 26 02:31:58 tapir modem-manager[13636]:   (ttyACM2) opening serial port...
May 26 02:31:58 tapir modem-manager[13636]:   (ttyACM1): using PDU mode for SMS
May 26 02:31:58 tapir modem-manager[13636]:   Modem /org/freedesktop/ModemManager/Modems/0: state changed (enabling -> enabled)
May 26 02:31:58 tapir NetworkManager[13638]:  WWAN now enabled by management service
May 26 02:31:59 tapir modem-manager[13636]:   Modem /org/freedesktop/ModemManager/Modems/0: state changed (enabled -> searching)
May 26 02:32:02 tapir modem-manager[13636]:   Modem /org/freedesktop/ModemManager/Modems/0: state changed (searching -> registered)
May 26 02:32:42 tapir NetworkManager[13638]:  Activation (wwan0) starting connection 'T-Mobile Default'
May 26 02:32:42 tapir NetworkManager[13638]:  (wwan0): device state change: disconnected -> prepare (reason 'none') [30 40 0]
May 26 02:32:42 tapir NetworkManager[13638]:  Activation (wwan0) Stage 1 of 5 (Device Prepare) scheduled...
May 26 02:32:42 tapir NetworkManager[13638]:  Activation (wwan0) Stage 1 of 5 (Device Prepare) started...
May 26 02:32:42 tapir NetworkManager[13638]:  Activation (wwan0) Stage 1 of 5 (Device Prepare) complete.
May 26 02:32:42 tapir modem-manager[13636]:   Modem /org/freedesktop/ModemManager/Modems/0: state changed (registered -> connecting)
May 26 02:32:44 tapir modem-manager[13636]:   Modem /org/freedesktop/ModemManager/Modems/0: state changed (connecting -> connected)
May 26 02:32:44 tapir NetworkManager[13638]:  Activation (wwan0) Stage 2 of 5 (Device Configure) scheduled...
May 26 02:32:44 tapir NetworkManager[13638]:  Activation (wwan0) Stage 2 of 5 (Device Configure) starting...
May 26 02:32:44 tapir NetworkManager[13638]:  (wwan0): device state change: prepare -> config (reason 'none') [40 50 0]
May 26 02:32:44 tapir NetworkManager[13638]:  (wwan0): bringing up device.
May 26 02:32:44 tapir NetworkManager[13638]:  Activation (wwan0) Stage 2 of 5 (Device Configure) successful.
May 26 02:32:44 tapir NetworkManager[13638]:  Activation (wwan0) Stage 3 of 5 (IP Configure Start) scheduled.
May 26 02:32:44 tapir NetworkManager[13638]:  Activation (wwan0) Stage 2 of 5 (Device Configure) complete.
May 26 02:32:44 tapir avahi-daemon[657]: Joining mDNS multicast group on interface wwan0.IPv6 with address fe80::dcc9:44ff:fe89:56fc.
May 26 02:32:44 tapir avahi-daemon[657]: New relevant interface wwan0.IPv6 for mDNS.
May 26 02:32:44 tapir avahi-daemon[657]: Registering new address record for fe80::dcc9:44ff:fe89:56fc on wwan0.*.
May 26 02:32:44 tapir NetworkManager[13638]:  Activation (wwan0) Stage 3 of 5 (IP Configure Start) started...
May 26 02:32:44 tapir NetworkManager[13638]:  (wwan0): device state change: config -> ip-config (reason 'none') [50 70 0]
May 26 02:32:44 tapir NetworkManager[13638]:  Activation (wwan0) Beginning DHCPv4 transaction (timeout in 45 seconds)
May 26 02:32:44 tapir NetworkManager[13638]:  dhclient started with pid 13689
May 26 02:32:44 tapir NetworkManager[13638]:  Activation (wwan0) Stage 4 of 5 (IPv6 Configure Timeout) scheduled...
May 26 02:32:44 tapir NetworkManager[13638]:  Activation (wwan0) Stage 3 of 5 (IP Configure Start) complete.
May 26 02:32:44 tapir NetworkManager[13638]:  Activation (wwan0) Stage 4 of 5 (IPv6 Configure Timeout) started...
May 26 02:32:44 tapir NetworkManager[13638]:  Activation (wwan0) Stage 4 of 5 (IPv6 Configure Timeout) complete.
May 26 02:32:44 tapir dhclient: Internet Systems Consortium DHCP Client 4.2.4
May 26 02:32:44 tapir dhclient: Copyright 2004-2012 Internet Systems Consortium.
May 26 02:32:44 tapir dhclient: All rights reserved.
May 26 02:32:44 tapir dhclient: For info, please visit https://www.isc.org/software/dhcp/
May 26 02:32:44 tapir dhclient: 
May 26 02:32:44 tapir NetworkManager[13638]:  (wwan0): DHCPv4 state changed nbi -> preinit
May 26 02:32:44 tapir dhclient: Listening on LPF/wwan0/de:c9:44:89:56:fc
May 26 02:32:44 tapir dhclient: Sending on   LPF/wwan0/de:c9:44:89:56:fc
May 26 02:32:44 tapir dhclient: Sending on   Socket/fallback
May 26 02:32:44 tapir dhclient: DHCPDISCOVER on wwan0 to 255.255.255.255 port 67 interval 3 (xid=0x40900a61)
May 26 02:32:47 tapir dhclient: DHCPDISCOVER on wwan0 to 255.255.255.255 port 67 interval 8 (xid=0x40900a61)
May 26 02:32:55 tapir dhclient: DHCPDISCOVER on wwan0 to 255.255.255.255 port 67 interval 13 (xid=0x40900a61)
ay 26 02:33:08 tapir dhclient: DHCPDISCOVER on wwan0 to 255.255.255.255 port 67 interval 7 (xid=0x40900a61)
May 26 02:33:15 tapir dhclient: DHCPDISCOVER on wwan0 to 255.255.255.255 port 67 interval 13 (xid=0x40900a61)
May 26 02:33:28 tapir dhclient: DHCPDISCOVER on wwan0 to 255.255.255.255 port 67 interval 20 (xid=0x40900a61)
May 26 02:33:29 tapir NetworkManager[13638]:  (wwan0): DHCPv4 request timed out.
May 26 02:33:29 tapir NetworkManager[13638]:  (wwan0): canceled DHCP transaction, DHCP client pid 13689
May 26 02:33:29 tapir NetworkManager[13638]:  Activation (wwan0) Stage 4 of 5 (IPv4 Configure Timeout) scheduled...
May 26 02:33:29 tapir NetworkManager[13638]:  Activation (wwan0) Stage 4 of 5 (IPv4 Configure Timeout) started...
May 26 02:33:29 tapir NetworkManager[13638]:  (wwan0): device state change: ip-config -> failed (reason 'ip-config-unavailable') [70 120 5]
May 26 02:33:29 tapir NetworkManager[13638]:  Activation (wwan0) failed for connection 'T-Mobile Default'
May 26 02:33:29 tapir NetworkManager[13638]:  Activation (wwan0) Stage 4 of 5 (IPv4 Configure Timeout) complete.
May 26 02:33:29 tapir modem-manager[13636]:   Modem /org/freedesktop/ModemManager/Modems/0: state changed (connected -> disconnecting)
May 26 02:33:29 tapir NetworkManager[13638]:  (wwan0): device state change: failed -> disconnected (reason 'none') [120 30 0]
May 26 02:33:29 tapir NetworkManager[13638]:  (wwan0): deactivating device (reason 'none') [0]
May 26 02:33:29 tapir avahi-daemon[657]: Interface wwan0.IPv6 no longer relevant for mDNS.
May 26 02:33:29 tapir avahi-daemon[657]: Leaving mDNS multicast group on interface wwan0.IPv6 with address fe80::dcc9:44ff:fe89:56fc.
May 26 02:33:29 tapir avahi-daemon[657]: Withdrawing address record for fe80::dcc9:44ff:fe89:56fc on wwan0.
May 26 02:33:29 tapir modem-manager[13636]:   Modem /org/freedesktop/ModemManager/Modems/0: state changed (disconnecting -> registered)

WTF?! I have asked myself?

After some time spent with the change-logs, debugging and asking google I have found this: https://bugs.launchpad.net/ubuntu/+source/network-manager/+bug/1172178

So it seems there is another pile of code in Linux kernel that aims to standardize some drivers and set common APIs, but it breaks things in meantime. Why it is always me who have to suffer this?:-)

Advice from Oleg Cherkasov at the end of the ticked worked for me. Thanks!

Wednesday, January 9, 2013

Shameful spyware in Smooth Gestures

Once again there was an ugly shameful and disgusting breach of what modern computing with OSS stands for. The Smooth Gestures extension to Chromium / Google Chrome was mucked with spyware which reports all the browsing actions to the creator of the extension.

Look here: http://codeonfire.cthru.biz/?p=96

I personally used the extension so I was a bit upset when it disappeared from Chrome Market but a bit of googling was enough to wake up. It said: What the fuck?!

Anyway I hunted for some replacement which wasn't that easy... But I have found one: Recognize it for Chrome - https://chrome.google.com/webstore/detail/recognize-it-for-chrome/bclaagbljldlbmihblajinlijckggkea

Sunday, December 30, 2012

OpenWRT insanity, shame and ignorancy

I have been hit by a idiotic, unreasonable, insane and stupid setting in OpenWRT:

/etc/sysctl.conf:
net.ipv4.netfilter.ip_conntrack_tcp_timeout_established=3600

Well what does it mean? Simple thing: Every long-lasting connections breaks after one hour. No matter whether I have TCP keepalive on, because TCP keepalive timeout defaults to 7200 seconds.

In practice you will see this:
brill@tapir ~ $ ssh milhouse.backbone.ignum.cz
Linux milhouse 2.6.32-5-amd64 #1 SMP Wed May 18 23:13:22 UTC 2011 x86_64

Keep keep your hands off this server!
Nikdo tu na nic nesahejte!

Last login: Sun Dec 30 00:46:47 2012 from 89.177.24.237
<no activity for more than 1 hour>
brill@milhouse:~$ Write failed: Broken pipe

brill@tapir ~ $

There was a flame on this topic on OpenWRT forum:

https://dev.openwrt.org/ticket/5777

And the result is: invalid, wontfix, fuckyourself.

Motherfuckers. They break everything. God... You can sacrifice filesystem, give up serial port, GPIO, LED diodes, whatever, but keep firewall working normally when you are building network appliance you idiots!

Thursday, April 12, 2012

Buggy Xfce xkb applet looses keyboard layout config

I have been hit by a nasty Xfce xkb applet bug, which is widely known in several bloody bugzillas (of RH/CentOS, Ubuntu, Debian,...) but it seems unresolved yet. The backround of the bug is, that the applet looses it's configuration, especially configured keyboard layouts and shortcut for switching them while for while which may seem to be random or it may seem to have some coincidence with suspending/wakeups of the computer. Well it is not random, it resolves to connections and disconnections of external (or even internal) USB keyboard, which may of course be triggered by suspend/wakeup.

So it seems I have the reason isolated. Now what is the solution? Well the applet is part of so-called Xfce-goodies, which is sort of external project. I did not find corresponding bugzilla nor some mailinglist etc. But I think that author is perhaps aware of this behavior, so there is no need to shout at him. But I needed the workaround and I think I have found one. It is simple and it is Linux-like: Just configure all your keyboards in xorg.xonf file in InputClass section and then use the applet as an indicator and switcher.

I am using following snippet of xorg.conf on my ThinkPad X301 which I am connecting and disconnecting to a USB keyboard:

Section "InputClass"
    Identifier "keyboard defaults"
    MatchIsKeyboard "on"
    Option "XkbLayout" "us, cz"
    Option "XkbVariant" ", qwerty_bksl"
    Option "XKbOptions" "grp:alt_shift_toggle, terminate:ctrl_alt_bksp"
EndSection

Desktop hell

Well after some time I am back with my reckless criticism of insane (but OpenSource) desktops. Basically what can you choose when you want a Linux desktop? Well, of course plenty of things but when you need getting things done and you do not want to play with own xrandr, xkb, xinput scripting or with printing, keybindings for multimedia keys etc. you would probably need some desktop environment. And there are quite a lot of such projects (look at Wikipedia). KDE. Gnome3. Xfce, LXDE, Unity to name some.

Well what you can find is that there is a huge hare-core of Gnome3 and Unity. People just hate them because these desktops are everything but useful and intuitive. The both are resembling some of iPad madness and with Unity there are rumors on the web that it is actually designed for tablet computers and its usage on desktop is some sort of side-effect of development in the meantime when Canonical (author of Unity & Ubuntu) is negotiating deals with tablet manufacturers. Pity. In fact these desktops forces you to do things by their ways (different from what we all are used to from previous generations of desktops starting from Win 3.11 up to Gnome 2.32) and they are putting obstacles between you and your productivity apps. And I concur with all these objections against particularly these two desktops.

There is of course KDE. KDE is way long from what is my idea of a decent desktop. I like some aspects of it but it lacks what I need - really fast desktop (with or rather without eye-candy but really fast!), easy-to used virtual desktops, app panel and good application switcher and support for docking/undocking (which means changing screen resolution, switching on/off the LVDS etc. It takes me to the remaining Xfce.

But current stable Xfce (on Debian Wheezy) to be specific is fare from being easy to use. I had to tweak it deeply to become usable. I had 4 most severe problems I had to cope with. NetworkManager (+ NM Applet) was unable to connect to set network. (Solved). There are nasty sounds in different applications set. The most annoying are terminal bell in gnome-terminal and gdm3 greeter sound (turned off easily). Two problems are more complicated and I am still working on them. First is Xfce xkb applet which looses it's configuration (I mean set layouts, layout-switcher keyboard shortcut etc.) each and every time you connect or disconnect USB keyboard. Which effectively means it looses the configuration each time you dock or undock your laptop. And the last problem is automatic switching of desktop resolution and putting LVDS to on/off state according to presence of another screen connected via Display Port. I have done some scripting workarounds to switch this semi-automatically but I am not proud of it at all.

Next time I am going to put here mentioned Debian tweaks for the two already-solved problems on my Debian Wheezy + Xfce desktop.

Tuesday, September 27, 2011

hpacucli: Error: No controllers detected. with hpsa and SmartArray P410i

I have encountered a fucking weird problem with hpacucli (HP utility for their RAID controllers) on ProLiant DL360 G7 or whatever. The controller was HP SmartArray P410i and the problem was simple:

root@XYZ:~# hpacucli 
HP Array Configuration Utility CLI 8.70-8.0
Detecting Controllers...Done.
Type "help" for a list of supported commands.
Type "exit" to close the console.

=> ctrl all show

Error: No controllers detected.

The solution is: Load the sg driver and that's it:

root@XYZ:~# modprobe sg
root@XYZ:~# hpacucli 
HP Array Configuration Utility CLI 8.70-8.0
Detecting Controllers...Done.
Type "help" for a list of supported commands.
Type "exit" to close the console.

=> ctrl all show

Smart Array P410i in Slot 0 (Embedded)    (sn: 500143801630C980)