Wednesday, 2021-02-17

*** zbr has quit IRC00:33
*** zbr has joined #openstack-infra00:45
*** dviroel has quit IRC00:59
*** rlandy has quit IRC01:24
*** ricolin has joined #openstack-infra01:29
*** zbr7 has joined #openstack-infra02:23
*** zbr has quit IRC02:23
*** zbr7 is now known as zbr02:23
*** rcernin has quit IRC02:27
*** rcernin has joined #openstack-infra03:07
*** psachin has joined #openstack-infra03:41
*** yamamoto has quit IRC04:02
*** ricolin has quit IRC04:16
*** yamamoto has joined #openstack-infra04:23
*** vishalmanchanda has joined #openstack-infra04:36
*** hamalq has quit IRC04:41
*** gyee has quit IRC05:01
*** zbr has quit IRC06:13
*** zbr has joined #openstack-infra06:14
*** ricolin has joined #openstack-infra06:29
*** mnaser has quit IRC06:32
*** mnaser has joined #openstack-infra06:34
*** ralonsoh has joined #openstack-infra06:41
*** slaweq has joined #openstack-infra07:07
*** jcapitao|off has joined #openstack-infra07:16
*** jcapitao|off is now known as jcapitao07:19
*** piotrowskim has joined #openstack-infra07:26
*** dklyle has quit IRC07:38
*** rcernin has quit IRC07:47
*** sboyron has joined #openstack-infra07:56
*** amoralej|off is now known as amoralej08:07
*** ricolin has quit IRC08:08
*** rpittau|afk is now known as rpittau08:12
*** ricolin_ has joined #openstack-infra08:12
*** rcernin has joined #openstack-infra08:12
*** andrewbonney has joined #openstack-infra08:13
*** ricolin_ is now known as ricolin08:17
*** rcernin has quit IRC08:19
*** eolivare has joined #openstack-infra08:21
*** przemeklal has quit IRC08:25
*** rcernin has joined #openstack-infra08:26
*** rcernin has quit IRC08:31
*** rcernin has joined #openstack-infra08:37
*** gfidente has joined #openstack-infra08:44
*** xek has joined #openstack-infra08:49
*** ysandeep|away is now known as ysandeep|ruck08:50
*** rcernin has quit IRC08:51
*** rcernin has joined #openstack-infra08:56
*** cloudnull has quit IRC08:59
*** jpena|off is now known as jpena08:59
*** cloudnull has joined #openstack-infra09:02
*** rcernin has quit IRC09:02
*** yamamoto has quit IRC09:04
*** yamamoto has joined #openstack-infra09:05
*** yamamoto has quit IRC09:05
*** ociuhandu has joined #openstack-infra09:07
*** yamamoto has joined #openstack-infra09:07
*** yamamoto has quit IRC09:07
*** yamamoto has joined #openstack-infra09:08
*** yamamoto has quit IRC09:08
*** yamamoto has joined #openstack-infra09:08
*** lucasagomes has joined #openstack-infra09:12
*** tosky has joined #openstack-infra09:14
*** ociuhandu has quit IRC09:17
*** tosky_ has joined #openstack-infra09:20
*** tosky has quit IRC09:24
*** tosky_ is now known as tosky09:25
*** rcernin has joined #openstack-infra09:28
*** rcernin has quit IRC09:35
*** yamamoto has quit IRC09:37
*** cloudnull has quit IRC09:37
*** derekh has joined #openstack-infra09:39
*** cloudnull has joined #openstack-infra09:40
*** ociuhandu has joined #openstack-infra09:47
*** rcernin has joined #openstack-infra09:59
*** dtantsur|afk is now known as dtantsur10:01
*** yamamoto has joined #openstack-infra10:07
*** ociuhandu has quit IRC10:10
*** ociuhandu has joined #openstack-infra10:20
*** rferraz has joined #openstack-infra10:30
*** rcernin has quit IRC10:31
*** yamamoto has quit IRC10:40
*** prometheanfire has quit IRC10:42
noonedeadpunkfungi: nope, we don't have encrypted storage atm and I'm right now about to setup it. But it will be totally opt-in, so smth that won't be enabled for all by default10:53
noonedeadpunkso unless you pick up luks encrypted volume type, it won't be encrypted10:54
*** yamamoto has joined #openstack-infra11:02
*** prometheanfire has joined #openstack-infra11:02
*** yamamoto has quit IRC11:02
*** dchen has quit IRC11:03
*** yamamoto has joined #openstack-infra11:04
*** yamamoto has quit IRC11:09
*** yamamoto has joined #openstack-infra11:10
*** sshnaidm has quit IRC11:15
*** yamamoto has quit IRC11:15
*** sshnaidm has joined #openstack-infra11:18
*** yamamoto has joined #openstack-infra11:21
*** yamamoto has quit IRC11:22
*** yamamoto has joined #openstack-infra11:23
*** yamamoto has quit IRC11:23
*** yamamoto has joined #openstack-infra11:23
*** jcapitao is now known as jcapitao_lunch11:26
*** yamamoto has quit IRC11:28
*** yamamoto has joined #openstack-infra11:35
*** yamamoto has quit IRC11:35
*** yamamoto has joined #openstack-infra11:35
*** dviroel has joined #openstack-infra11:37
*** ociuhandu has quit IRC11:53
*** eolivare has quit IRC12:09
*** fdegir5 is now known as fdegir12:24
*** jpena is now known as jpena|lunch12:30
*** rlandy has joined #openstack-infra12:31
*** yamamoto has quit IRC12:39
*** rcernin has joined #openstack-infra12:58
*** rcernin has quit IRC13:02
*** amoralej is now known as amoralej|lunch13:04
*** jcapitao_lunch is now known as jcapitao13:07
*** d34dh0r53 has joined #openstack-infra13:08
*** yamamoto has joined #openstack-infra13:15
*** eolivare has joined #openstack-infra13:19
*** jpena|lunch is now known as jpena13:23
*** nweinber has joined #openstack-infra13:26
*** yamamoto has quit IRC13:28
*** amoralej|lunch is now known as amoralej14:00
*** ysandeep|ruck is now known as ysandeep|mtg14:01
*** hemna has quit IRC14:12
*** hemna has joined #openstack-infra14:13
*** psachin has quit IRC14:18
*** lpetrut has joined #openstack-infra14:20
funginoonedeadpunk: thanks, just trying to rule out things factors which might cause growroot to randomly fail in citycloud14:30
funginow that we should be collecting early boot messages from growroot operations, we can hopefully investigate further14:30
noonedeadpunkhm, can't think even about reasons why this might happen...14:31
fungiyeah, some small percentage of nodes we boot there are winding up with a rootfs roughly the size of our images rather than the size specified in the flavor, but ansible sees the block device is the correct size, so we're suspecting something sometimes going wrong with growroot14:34
fungibut we've only seen it happen on citycloud14:34
noonedeadpunkAnd I think images are build with nodepool?14:35
fungiyes, the image is built once and uploaded to all providers14:39
fungiso it's the same image booting normally in other providers (and also booting normally in citycloud most of the time)14:40
noonedeadpunkI can imagine some issues with metadata proxy but then we wouldn't be able to access vm at all...14:41
fungiand we use configdrive anyway14:41
noonedeadpunkah14:41
*** zzzeek has quit IRC14:49
*** zzzeek has joined #openstack-infra14:50
*** jhesketh has quit IRC14:52
*** slaweq has quit IRC15:00
*** ociuhandu has joined #openstack-infra15:01
*** slaweq has joined #openstack-infra15:04
*** ociuhandu has quit IRC15:05
*** ociuhandu has joined #openstack-infra15:06
*** ociuhandu has quit IRC15:15
*** ociuhandu has joined #openstack-infra15:16
*** ociuhandu_ has joined #openstack-infra15:19
*** ociuhandu_ has quit IRC15:19
*** ociuhandu_ has joined #openstack-infra15:20
*** ociuhandu has quit IRC15:21
*** yamamoto has joined #openstack-infra15:25
*** yamamoto has quit IRC15:36
*** dklyle has joined #openstack-infra15:41
*** hemna has quit IRC15:41
*** hemna has joined #openstack-infra15:43
*** ysandeep|mtg is now known as ysandeep|ruck15:44
*** ysandeep|ruck is now known as ysandeep|dinner16:08
*** dklyle has quit IRC16:21
*** lpetrut has quit IRC16:25
*** dklyle has joined #openstack-infra16:29
*** hashar has joined #openstack-infra16:29
*** hemna has quit IRC16:31
*** hemna has joined #openstack-infra16:32
*** piotrowskim has quit IRC16:32
*** hashar has quit IRC16:35
*** hashar has joined #openstack-infra16:38
*** slaweq has quit IRC16:53
*** rcernin has joined #openstack-infra16:58
*** ociuhandu_ has quit IRC17:02
*** rcernin has quit IRC17:02
*** ociuhandu has joined #openstack-infra17:02
*** lucasagomes has quit IRC17:03
*** jpena is now known as jpena|brb17:14
*** dtantsur is now known as dtantsur|afk17:17
*** amoralej is now known as amoralej|off17:17
*** eolivare has quit IRC17:28
*** ysandeep|dinner is now known as ysandeep|ruck17:29
*** yamamoto has joined #openstack-infra17:32
*** jcapitao has quit IRC17:33
*** jpena|brb is now known as jpena17:36
*** yamamoto has quit IRC17:37
*** rpittau is now known as rpittau|afk17:41
*** gyee has joined #openstack-infra17:43
*** cloudnull has quit IRC17:49
*** cloudnull has joined #openstack-infra17:49
*** ociuhandu has quit IRC17:54
*** ralonsoh has quit IRC17:54
*** gfidente is now known as gfidente|afk18:03
*** rferraz has quit IRC18:08
*** ysandeep|ruck is now known as ysandeep|away18:10
*** derekh has quit IRC18:11
*** jpena is now known as jpena|off18:12
*** auristor has quit IRC18:21
*** slaweq has joined #openstack-infra18:23
*** andrewbonney has quit IRC18:33
*** auristor has joined #openstack-infra18:36
fungiamorin: we stopped using swift in ovh bhs because of this incident, and are wondering if it's safe to start using it again given the incident was not marked resolved (connectivity seems to be restored at least): http://travaux.ovh.net/?do=details&id=4899719:08
*** slaweq has quit IRC19:20
*** yamamoto has joined #openstack-infra19:33
*** yamamoto has quit IRC19:38
*** elod has quit IRC20:17
*** elod has joined #openstack-infra20:19
*** hashar has quit IRC20:23
rledisezhi fungi. I'm sorry that you were impacted by this network incident. I don't have specific information about this incident, but as you probably know, situation in Texas is still complicated. If there is no emergency, I would wait that the incident is marked as fully resolved. If it is urgent for you, I can go ask some network colleagues how confident they are currently.20:24
fungirledisez: no worries, just didn't know if somebody forgot to resolve the incident last week, we're happy to keep waiting20:24
fungiand yeah, the net result was that routes between rackspace's dfw region and ovh's bhs region were a black hole, with traceroutes in both directions dying at ovh's dfw border20:26
fungithat seems to have been fixed, we're just hesitant to assume it won't go back down if the incident itself is marked as ongoing20:26
*** xek has quit IRC20:27
clarkband ya probably bestthat we don't push things in texas right now. It sounds really unpleasant for many and their physical safety is much more important right now20:29
fungii concur20:29
*** xek has joined #openstack-infra20:31
*** xek has quit IRC20:37
*** xek has joined #openstack-infra20:37
*** xek has quit IRC20:41
*** xek has joined #openstack-infra20:42
*** xek has quit IRC20:44
*** slaweq has joined #openstack-infra20:46
*** rcernin has joined #openstack-infra20:58
*** rcernin has quit IRC21:03
*** rcernin has joined #openstack-infra21:22
*** cloudnull has quit IRC21:28
*** cloudnull has joined #openstack-infra21:33
*** yamamoto has joined #openstack-infra21:34
*** vishalmanchanda has quit IRC21:37
*** yamamoto has quit IRC21:39
*** eandersson has quit IRC21:40
*** gmann is now known as gmann_afk21:51
*** rcernin has quit IRC21:53
*** rcernin has joined #openstack-infra22:00
*** slaweq has quit IRC22:02
*** yamamoto has joined #openstack-infra22:15
*** nweinber has quit IRC22:22
*** jhesketh has joined #openstack-infra22:32
*** sboyron has quit IRC23:07
*** rlandy is now known as rlandy|bbl23:11
*** jistr has quit IRC23:20
*** jistr has joined #openstack-infra23:21
*** gmann_afk is now known as gmann23:22
*** prometheanfire has quit IRC23:27
*** prometheanfire has joined #openstack-infra23:51

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!