Struggling with new NVIDIA Mellanox switches not passing traffic? You’re not alone; the default ‘Hybrid Port’ mode is the likely culprit, and here’s how to fix it for good.

I Feel Lost With NVIDIA Mellanox Switches: A Senior Engineer’s Guide

It was 3 AM. The PagerDuty alert was screaming about a full-stack outage. Our brand new, high-performance storage cluster, connected with shiny new NVIDIA SN2010 switches, had just gone completely dark. We’d spent all day racking it, cabling it, and getting green lights everywhere. But not a single packet was flowing. The junior on call was convinced the optics were bad or the cables were faulty. I knew better. I’d seen this ghost before, and its name is “Hybrid Port Mode.” If you’re pulling your hair out wondering why your simple Layer 2 network isn’t working on one of these switches, take a breath. You’re not crazy, you’ve just stumbled into one of the most common “gotchas” in the data center.

The “Why”: It’s Not You, It’s the Default Mode

Here’s the deal. Most of us come from a world of Cisco, Arista, or Dell switches where a port is a Layer 2 access/trunk port by default. You plug something in, and it just works. NVIDIA (formerly Mellanox) switches, especially those running MLNX-OS, have a different philosophy. Out of the box, their ports are often in Hybrid Mode.

Think of Hybrid Mode as a weird mix of a Layer 2 trunk port and a Layer 3 routed port. It’s powerful for complex InfiniBand or RoCE deployments but absolutely maddening for a simple server or storage connection. The switch is expecting specific types of tagged and untagged traffic that your server’s NIC just isn’t sending, so it silently drops the packets. The link light is green, everything looks fine, but the network is a black hole. The root cause is that the switch isn’t configured to be the simple, “dumb” L2 switch you expect it to be.

Your Battle Plan: Three Ways to Tame the Beast

Don’t throw the switch out the window just yet. Here are three ways to fix this, ranging from a quick fix to get you online now to the permanent solution you should implement during your next maintenance window.

1. The Quick Fix: The Per-Port Hammer

You have a new server, `prod-db-01`, that needs to be online right now. You don’t have time to reboot the switch. This method surgically changes only the port you need, leaving the rest of the switch untouched.

Log into the switch’s CLI and run these commands. Let’s say your server is plugged into port 5.

enable
configure terminal
interface ethernet 1/5
switchport mode ethernet
exit
write memory

This command explicitly tells port 1/5 to stop being a confusing hybrid port and act like a standard Layer 2 Ethernet port. Traffic should start flowing immediately. You can also apply this to a range of ports to save time.

configure terminal
interface ethernet 1/1-1/16 switchport mode ethernet
exit
write memory

Warning: Don’t forget write memory! If you don’t save the configuration, your changes will disappear the next time the switch reboots, and you’ll be having this exact same conversation with yourself in six months. Trust me.

2. The Permanent Fix: The “Right Way”

Doing the per-port fix every time you plug in a new device is tedious and error-prone. The real, permanent solution is to change the entire system’s profile. This tells the switch to make all ports Ethernet by default, forever.

This is the command you want, but be aware: it requires a reboot.

enable
configure terminal
system profile ethernet-only
exit
write memory

After you run this and save the config, the switch will prompt you to reboot. Once it comes back up, it will behave like any other Layer 2 switch you’ve ever worked with. Every port will be in Ethernet mode by default. This is the fix we implement on every new NVIDIA switch we deploy at TechResolve. Do it during a planned maintenance window and save your future self a world of pain.

3. The ‘Nuclear’ Option: Factory Reset & Start Clean

Sometimes you inherit a switch from another team, or you’ve been trying so many different commands that you’re not even sure what the current state is. The configuration is a mess, and you just want to start from a clean slate. This is where the factory reset comes in.

This is the most disruptive option, as it will wipe out everything—VLANs, management IPs, everything. But sometimes, a clean slate is the fastest path forward.

CRITICAL: Before you do this, back up the current configuration if there’s anything you might need to reference later! Use show running-config and copy the output to a safe place.

enable
reload factory-default

The switch will ask for confirmation because this is a destructive action. After it reboots, it will be factory fresh. From there, you can immediately apply Fix #2 (the `system profile ethernet-only` command), set up your management IP, and build your configuration from a known-good state.

Comparing Your Options

To make it simple, here’s how the solutions stack up:

So, there you have it. That feeling of being lost is completely normal. These switches are incredibly powerful, but they don’t hold your hand. Now you have a clear plan. Go get that network traffic flowing.

Darian Vance

Lead Cloud Architect & DevOps Strategist

With over 12 years in system architecture and automation, Darian specializes in simplifying complex cloud infrastructures. An advocate for open-source solutions, he founded TechResolve to provide engineers with actionable, battle-tested troubleshooting guides and robust software alternatives.

🤖 Frequently Asked Questions