#10209 Investigate networking on worker01-worker03 in prod OCP clusters
Closed: Fixed 2 years ago by kevin. Opened 2 years ago by t0xic0der.

Describe what you would like us to do:

We need to make sure that they are all physically wired same and have the same configured on switches/routers.

The oshift-dell01 has only a single interface connected while the rest have two and oshift-dell03 does not seem to have network access in the second interface (eno34).

oshift-dell01             IN        A     10.3.160.180
oshift-dell02             IN        A     10.3.160.181
oshift-dell03             IN        A     10.3.160.182

When do you need this to be done by? (YYYY/MM/DD)

As soon as possible :)


Metadata Update from @mohanboddu:
- Issue priority set to: Waiting on Assignee (was: Needs Review)
- Issue tagged with: medium-gain, medium-trouble, ops

2 years ago

Are the two interfaces being bonded? ie, they should be on the same vlan?

Is that how staging workers are setup?

I don't think their bonded, but I'm not certain.

On staging we have 2 interfaces on each machine eno33 and eno34, they all have network access on the eno33 interfaces. We've disabled dhcp on all the eno34 interfaces.

On prod we've disabled dhcp on the eno33 interfaces:

  • worker01 has no eno33, and has network access on eno34
  • worker02 has both interfaces, and has network access on eno34
  • worker03 has both interfaces, and has no network access on eno34`

I looked into things and asked networking about them... will let you know when I get any answers. :)

So, worker01 may not have it's first port connected. Hopefully someone can check that next week.

Can you just use eno33 on worker03? Or they all have to be the same?

I currently see worker01/02 up but worker03 is not?

Are the 2 eno33 interfaces on the same vlan?
Are the 3 eno34 interfaces on the same vlan?

I guess you could make this work with either:

1) fixing eno33 on worker1 and switching to use eno33 on all 3
or
2) making sure all the eno34's are on the same vlan and using them?

Can you get the mac addresses for these interfaces and tell me what we need to change where to make this work?

Yep we can use en033 on worker03, but as you've said, we need them all to be configured the same. So if we can get eno33 connected on worker01, then we can use that interface on all workers which would be great!

I don't know which VLAN these interfaces are on, did a little googling but unsure how to check for sure!

Worker01:
- eno33 68:05:CA:CE:A3:C8
- eno34 68:05:CA:CE:A3:C9

Worker02:
- eno33 68:05:CA:CE:A2:90
- eno34 68:05:CA:CE:A2:91

Worker03:
- eno33 68:05:CA:CE:A7:4C
- eno34 68:05:CA:CE:A7:4D

So, the fix for eno33 on worker01 is likely going to requre a on-site person. The next visit is in October sometime... :(

So, can you just make it work for now by using eno34 on 01 and 02 and eno33 on 03 for now?

Hmm the problem comes with machine configs, and if we can't apply one to all the workers that fits, eg, we need to disable dhcp on one of the interfaces.

If we can get eno33 wired up on worker01, we can disable eno34 on all workers, or if we can figure out why we can't get network access on eno34 on worker03, the current config deployed will work fine.

I'll see if I can figure out how to add worker03, with eno33 in use, and exclude it from using the current machineconfig for workers..

So, I asked about eno34 on worker03... "68:05:CA:CE:A7:4D is already on 703" (703 is the 10.3.163.x prod vlan) so it should work I would think?

This seems to have been sorted out now.

Metadata Update from @kevin:
- Issue close_status updated to: Fixed
- Issue status updated to: Closed (was: Open)

2 years ago

Thanks @kevin ! yes it is working great now! :v:

Login to comment on this ticket.

Metadata
Boards 1
ops Status: Done