Troubleshooting
RouterOS User Manager in Routed Networks
Troubleshooting RouterOS User Manager RADIUS server in routed networks: discovering that RADIUS replies sourced from loopback interfaces are rejected by clients, requiring RADIUS server to use interface IPs or non-loopback addresses for proper operation.Duplicate MAC Address Madness
Troubleshooting duplicate MAC addresses in redundant firewall HA deployments: discovering non-unique VRRP group IDs across sites caused site-to-site traffic loss, with MAC addresses appearing on unexpected switch ports during failover events.Syncing RouterOS address-lists
Pull-based RouterOS address-list synchronization using dynamic webserver scripts: generating conditional add/update commands from database backend, fetching and executing scripts on multiple routers, tracking updates with timestamps, and avoiding API version compatibility issues.Troubleshooting a Duplicate OSPF Router ID
Troubleshooting OSPF duplicate router IDs in VPLS networks: discovering 10-second route flapping caused by cloned PE router configuration, identifying router ID duplication, and understanding network-wide effects of duplicate router identifiers in MPLS service provider deployments.Inadvertent Transit
Analyzing BGP route leak causing AS7122 outage: AS53443 leaked 535,292 DFZ routes from AS6327 during second provider turnup due to missing outbound prefix filters, combined with AS7122 accepting routes without customer filters, congesting 70% of internet-bound traffic for 13 minutes.fpinglog
Logging continuous ping latency and packet loss for troubleshooting using fping -D -l flags: capturing unix timestamps with latency measurements, converting timestamps with Perl, and logging output to file for later analysis.ESXi 6up2
VMware ESXi 6.0 Update 2 improvements: using HTML5 VMware Host Client for full VM provisioning without Windows vSphere client, functional browser-based console, fixed macOS Remote Console launching, and no Flash required for administration.IXP Peering Lan Hijacks
MBIX Peering LAN IP space hijacked via BGP and used for spam: investigating unauthorized BGP announcements of 206.72.208.0/24 through AS-Path 9002 44050 131788, demonstrating why IX peering LAN space should not be globally routable.Nagios and IPv6
Implementing IPv6 monitoring in Nagios: creating dual-stack service checks with -4/-6 flags, defining custom _ADDRESS4/_ADDRESS6 variables for DNS independence, and navigating IPv4-only plugin limitations like check_icmp.4 ASNs Disappear in Atomic Aggregate
AS7122 enables atomic aggregate on /16 announcements, accidentally suppressing four Manitoba customer ASNs (21876, 23001, 32433, 54937) from global routing table—analyzing route withdrawals via RIPE collectors and multihoming implications.Running Smokeping on Jessie
Fixing Smokeping after Debian Wheezy to Jessie upgrade: Apache 2.2 to 2.4 migration breaks conf.d structure, requiring manual symlinks in conf-enabled directory for smokeping and OTRS configurations.Nagios Logs, Handy One-Liner
Converting Nagios log timestamps to human-readable format using Perl one-liner: piping epoch timestamps through localtime() substitution for on-the-fly date conversion in monitoring logs and BGP data.Removing Stubborn Trash Items
Fixing macOS Yosemite trash items that won't delete: using CLI to remove immutable flags with chflags -R nouchg and force deletion when GUI methods fail.Latency and Effects on TCP Performance
Understanding Bandwidth Delay Product and Long Fat Networks: how RTT and TCP receive window size limit per-thread throughput, why download managers use multiple connections, and how TCP window auto-scaling optimizes performance.virtio NIC on OpenBSD under KVM
Fixing intermittent OpenBSD vio(4) network dropouts under KVM: disabling RingEventIdx via config -ef /bsd kernel flag adjustment resolves interface connectivity issues requiring periodic ifconfig cycling.Sleep Talking
Troubleshooting Intel I217-LM NIC bug causing 3300 pps IPv6 multicast storms during S1 sleep: using tcpdump/Wireshark to identify MLDv2 listener reports flooding network, resolved with driver v19.5 update.Does Everything Still Work?
Building test-driven infrastructure with Bash Automated Testing System (BATS): creating 27 pre-change validation tests for websites, redirects, SSL endpoints, and dynamic content to answer "does everything still work?"MikroTik SNMP Oddity
Troubleshooting MikroTik CCR1036 SNMP failure on loopback interface: packet captures reveal implementation quirk refusing to return traffic via different interface than request arrived on, requiring monitoring via closest interface IP.SaltStack, CouchDB, and Nagios
Building distributed monitoring with SaltStack returners to CouchDB: storing nagios check results globally from multiple VPS locations, visualizing traceroutes/latency, and designing contextual alerts beyond simple up/down states.Observium November Release
Observium 0.14.11 released on schedule: new alerting system, Cisco ASA IPv4 session graphing from FIREWALL-MIB, Cambium Canopy support, and enabling built-in VMware ESXi SNMP monitoring.Service Status via SaltStack 2014.7 with Nagios
Exploring SaltStack 2014.7's nagios module: remote execution of nagios-plugins on minions, pillar-defined check lists with Jinja templating, distributed monitoring ideas, and contributing documentation fixes to SaltStack.Troubleshooting ICMPv6 with tcpdump
Fixing OpenBSD PF firewall ICMPv6 rules by using tcpdump to identify required neighbor discovery types (133-137): isolating NDP traffic with packet filters and properly allowing router/neighbor solicitation/advertisement messages.SSLv3 Disabled
Responding to POODLE vulnerability by disabling SSLv3 across all services: enforcing TLSv1.0+, updating cipher suites from Mozilla wiki, logging protocol/cipher combinations to verify modern client support.Internet Connection Sharing
Tracking down rogue DHCP server on corporate network: using arpwatch to identify Mac sharing causing 192.168.137.x leases, correlating MAC to CDP/switch port, and why DHCP snooping prevents these issues.Firewall Log Stats
Analyzing OpenBSD PF firewall logs with enhanced Pantz PFlog Stats: added GeoIP support using Maxmind DB, updated whois links for global regions, and automated blocked packet analysis with Perl scripting.CVE-2014-6271 - ShellShock
ShellShock bash vulnerability (CVE-2014-6271) allowing remote code execution via crafted environment variables: patching Debian systems with SaltStack, tracking 6 related CVEs, and emergency response coordination.Strange SNMP SetRequest Traffic
Investigating spoofed SNMP SetRequest attacks using Google DNS IPs (8.8.8.8): 45-minute packet capture reveals sequential scanning patterns targeting ISP /16 blocks, confirming Internet Storm Center findings.Major Internet Issues Today
August 12, 2014: IPv4 routes breach 512k triggering router crashes, Shaw Canada experiences nationwide routing failures, and AS701/AS705 de-aggregation event cascades through unfiltered peers.BGP Hijacking In News Again
Canadian ISP hijacks 51 prefixes to redirect cryptocurrency miners via unfiltered AS6939 peer: bogon routes, more-specifics targeting Digital Ocean/AWS, and lessons in prefix filtering failures.pinglog
Shell scripts for ping troubleshooting: one detects IPv4/IPv6 connectivity failures with timeout alerts, another timestamps all ping output for detailed logging and analysis.Leaking Routes
Investigating a suspected BGP route leak involving TeraGo impacting AS paths between my networks. Traceroute analysis, AS-PATH inspection, and comparison of expected vs. leaked routes.1f j00 R R34d1N 7h12 m3m0ry j00 R pWN3D
Heartbleed OpenSSL vulnerability analysis. Memory disclosure vulnerability affecting OpenSSL 1.0.1-1.0.1f, widespread device impact, and scanning/remediation strategies.AS4761 April 2 2014 Prefix Origination Event
BGP prefix hijacking incident involving Indonesian ISP Indosat (AS4761) advertising multiple ISP prefixes. Analysis of Thai BGP upstream filtering failure and impact on global routing.XKCD Passwords
Examining XKCD comic strip #936 on password security and memorable passphrases. Review of xkcd-password implementations on GitHub for generating user-friendly passwords with entropy calculation.iPerf UDP
Using iperf for UDP performance testing to isolate VoIP-related network issues. Demonstrates measuring jitter, packet loss, and bandwidth with IPv6 over long distances.goto fail
Analysis of critical goto-related bugs in SSL/TLS signature verification in Apple and GnuTLS. Examines the dangers of goto statements and implications of delayed security patching.SSL Ciphers
SSL/TLS cipher and protocol configuration for nginx, dovecot, and Apache. Securing connections by selecting strong ciphers and disabling weak algorithms. Note: See Mozilla SSL Configuration Generator for current best practices."My Traffic Is Not Reaching The Destination"
Troubleshooting UDP stream connectivity issues using packet captures and Wireshark. Identified TTL expiration as root cause of packets not reaching destination across multiple hops.Implementing BCP38
Implementing BCP38 ingress filtering on Cisco routers to prevent spoofed traffic from leaving your network. Includes ACL examples to filter bogus source addresses and private ranges.SSL CLI Troubleshooting
Using openssl s_client to test SSL/TLS connections and verify certificate chains for encrypted email services like POP3, IMAP, and SMTP from the command line.