| pabelanger | do you guys have any functional tests running on fedora-24? | 00:25 |
|---|---|---|
| pabelanger | ianw: $ ls /proc/ | wc -l | 00:55 |
| pabelanger | 2241 | 00:55 |
| pabelanger | as you can imagine, kill_chroot_processes:141 is taking some time | 00:55 |
| pabelanger | is that step really needed? | 00:55 |
| ianw | hmm, i wouldn't think that would take that long? | 00:56 |
| ianw | is it blocking on something | 00:56 |
| ianw | that was a recent change where we check if any process is still running the chroot & try to kill it | 00:56 |
| pabelanger | nope, think it is just taking a while for each lookup | 00:56 |
| ianw | hence the name :) | 00:56 |
| ianw | so the problem might more be a stuck chroot process? | 00:56 |
| pabelanger | this is a osic VM, with alot of kworkers processes | 00:56 |
| pabelanger | ianw: best way to enable a debug? | 00:57 |
| ianw | just going back to look over that code again | 00:57 |
| pabelanger | just hacked in set -x for now | 00:58 |
| pabelanger | rebuilding | 00:58 |
| pabelanger | http://paste.openstack.org/show/592150/ | 00:59 |
| pabelanger | over and over | 00:59 |
| pabelanger | about a second for each loop | 00:59 |
| pabelanger | so, 2241 procs is 37mins | 01:00 |
| pabelanger | :( | 01:00 |
| ianw | that's way way too long | 01:01 |
| pabelanger | so, this is osic-cloud1-s3500 and fedora-24 | 01:02 |
| ianw | i guess it's a lot of forking | 01:03 |
| pabelanger | Ya, something is going on | 01:04 |
| pabelanger | is non-voting right now | 01:05 |
| pabelanger | so I'll poke more at it | 01:05 |
| ianw | we could save at least a bit of time by not looking at the name unconditionally | 01:07 |
| ianw | but still, "ps -aef" isn't slow, is it? | 01:08 |
| *** chhavi has joined #openstack-dib | 01:13 | |
| ianw | yeah, it's a slow loop | 01:17 |
| ianw | i wonder if this would be much faster in python | 01:22 |
| ianw | yeah, without forking, its *way* faster | 01:27 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder: Speed up chroot checking loop https://review.openstack.org/410000 | 01:49 |
| ianw | pabelanger: ^ something like that should work a lot better i think | 01:50 |
| *** chhavi has quit IRC | 02:14 | |
| *** chhavi has joined #openstack-dib | 02:23 | |
| *** chhavi has quit IRC | 02:46 | |
| *** sudipto has joined #openstack-dib | 03:07 | |
| *** sudipto_ has joined #openstack-dib | 03:10 | |
| *** sudipto has quit IRC | 03:12 | |
| *** pmannidi_ has joined #openstack-dib | 03:18 | |
| *** pmannidi has quit IRC | 03:19 | |
| *** sudipto has joined #openstack-dib | 03:23 | |
| *** sudswas__ has joined #openstack-dib | 03:23 | |
| *** sudipto_ has quit IRC | 03:24 | |
| *** sudipto has quit IRC | 03:24 | |
| *** sudswas__ has quit IRC | 03:24 | |
| *** sudipto has joined #openstack-dib | 03:24 | |
| *** sudipto_ has joined #openstack-dib | 03:24 | |
| *** sudipto_ has quit IRC | 03:32 | |
| *** sudipto has quit IRC | 03:32 | |
| *** sudipto_ has joined #openstack-dib | 03:35 | |
| *** sudipto has joined #openstack-dib | 03:35 | |
| *** sudswas__ has joined #openstack-dib | 03:45 | |
| *** sudswas__ has quit IRC | 03:45 | |
| *** sudipto has quit IRC | 03:46 | |
| *** sudipto_ has quit IRC | 03:47 | |
| *** sudipto has joined #openstack-dib | 03:49 | |
| *** sudipto_ has joined #openstack-dib | 03:49 | |
| *** sudipto_ has quit IRC | 04:05 | |
| *** sudipto has quit IRC | 04:05 | |
| *** sudipto has joined #openstack-dib | 04:30 | |
| *** sudipto_ has joined #openstack-dib | 04:30 | |
| *** sudipto_ has quit IRC | 05:14 | |
| *** sudipto has quit IRC | 05:14 | |
| *** jaianshu has joined #openstack-dib | 06:42 | |
| *** jaianshu_ has joined #openstack-dib | 06:43 | |
| *** jaianshu_ has left #openstack-dib | 06:43 | |
| *** hashar has joined #openstack-dib | 08:30 | |
| *** jaianshu_ has joined #openstack-dib | 09:00 | |
| *** jaianshu has quit IRC | 09:03 | |
| *** sudipto has joined #openstack-dib | 09:18 | |
| *** sudipto_ has joined #openstack-dib | 09:18 | |
| *** hashar has quit IRC | 09:20 | |
| *** hashar has joined #openstack-dib | 09:24 | |
| *** owalsh-afk is now known as owalsh | 09:27 | |
| *** jaianshu_ has quit IRC | 10:49 | |
| *** sudipto has quit IRC | 11:12 | |
| *** sudipto_ has quit IRC | 11:12 | |
| *** zhugaoxiao has joined #openstack-dib | 11:13 | |
| *** hashar is now known as hasharLunch | 11:21 | |
| *** jaianshu has joined #openstack-dib | 11:34 | |
| *** sudipto has joined #openstack-dib | 11:36 | |
| *** sudipto_ has joined #openstack-dib | 11:36 | |
| jaianshu | greghayens, ianw: would you please reook at https://review.openstack.org/#/c/385880/...i did had +2 from greghayens, after that i had uploaded a new patch for some minor comments and so require you to have a re-look. | 11:52 |
| *** zhugaoxiao has quit IRC | 11:56 | |
| *** zhugaoxiao has joined #openstack-dib | 11:56 | |
| *** jaianshu has quit IRC | 12:08 | |
| *** hasharLunch is now known as hashar | 12:39 | |
| pabelanger | ianw: let me test | 14:26 |
| *** brault_ has quit IRC | 15:42 | |
| *** dmarlin has joined #openstack-dib | 15:55 | |
| greghaynes | ianw: could I get some feedback on https://review.openstack.org/#/c/408288/ ? octavia folks are depending on that stack ATM | 15:58 |
| greghaynes | I think to unbreak some of their jos | 15:58 |
| greghaynes | jobs | 15:58 |
| openstackgerrit | greghaynes proposed openstack/diskimage-builder: Allow disabling manifests https://review.openstack.org/358865 | 16:25 |
| openstackgerrit | greghaynes proposed openstack/diskimage-builder: Allow disabling manifests https://review.openstack.org/358865 | 16:28 |
| openstackgerrit | Dirk Mueller proposed openstack/diskimage-builder: Switch to openSUSE Leap 42.2 release by default https://review.openstack.org/409071 | 16:49 |
| *** hashar has quit IRC | 16:54 | |
| *** strigazi is now known as strigazi_AFK | 17:07 | |
| pabelanger | Finally have some spare time to try and get docker images going again with DIB | 18:09 |
| pabelanger | lets see how this magic works | 18:09 |
| greghaynes | oh nice | 18:37 |
| pabelanger | $ sudo docker images | 18:40 |
| pabelanger | REPOSITORY TAG IMAGE ID CREATED SIZE | 18:40 |
| pabelanger | docker-ubuntu latest 184df7971312 About a minute ago 679.5 MB | 18:40 |
| pabelanger | kinda fat | 18:40 |
| pabelanger | but worked | 18:40 |
| pabelanger | ubuntu-minimal | 18:40 |
| pabelanger | now, to use my simply-playbook element to overlay something | 18:41 |
| greghaynes | oh what | 18:41 |
| greghaynes | thats stupid big | 18:41 |
| greghaynes | are you using the latest dib release? | 18:41 |
| pabelanger | yes | 18:41 |
| greghaynes | hrm | 18:41 |
| pabelanger | right from master | 18:42 |
| greghaynes | there was a fix that should have made it much smaller | 18:42 |
| pabelanger | actualkly | 18:42 |
| pabelanger | actually( | 18:42 |
| pabelanger | let me check | 18:42 |
| pabelanger | yup: 8c74c8e409adf21094a5973bef84a02ba8876464 | 18:42 |
| greghaynes | huh | 18:43 |
| greghaynes | and just ubuntu-minimal, not any extra stuff? | 18:43 |
| greghaynes | I wonder if something related to how to do the tar'ing is making it explode in size | 18:43 |
| pabelanger | disk-image-create -x -t docker,tar --checksum --no-tmpfs --qemu-img-options 'compat=0.10' -o ubuntu-trusty ubuntu-minimal --docker-target=docker-ubuntu | 18:43 |
| pabelanger | was the command | 18:43 |
| pabelanger | trying again, cause I actually wanted trusty | 18:43 |
| greghaynes | is the tar roughly the same size? | 18:44 |
| pabelanger | will know in a min | 18:44 |
| greghaynes | ok | 18:44 |
| pabelanger | wow | 18:48 |
| pabelanger | it is fat too | 18:48 |
| pabelanger | -rw-r--r-- 1 pabelanger pabelanger 576M Dec 13 18:48 ubuntu-trusty.tar | 18:48 |
| pabelanger | docker-ubuntu latest 4faadc4b1094 32 seconds ago 568 MB | 18:49 |
| greghaynes | womp womp | 18:49 |
| greghaynes | might just be ubuntu then. I thought I had them down to about 300M | 18:49 |
| pabelanger | oh, I think I know | 18:49 |
| pabelanger | ya, /lib is 300M itself | 18:51 |
| greghaynes | nice. Theres a deal I was doing to sort installed dpkg's by size | 18:52 |
| pabelanger | I suspect, if I create an ubuntu-container element, that strips out the kernel / modules, I'll save a few hundred mbs | 19:27 |
| *** sudipto has quit IRC | 19:31 | |
| *** sudipto_ has quit IRC | 19:31 | |
| pabelanger | -rw-r--r-- 1 pabelanger pabelanger 245M Dec 13 19:43 ubuntu-trusty.tar | 19:43 |
| pabelanger | without kernel | 19:43 |
| pabelanger | still fat, but less of it | 19:44 |
| pabelanger | down to 211MB if I drop base | 20:01 |
| openstackgerrit | Dirk Mueller proposed openstack/diskimage-builder: Switch to openSUSE Leap 42.2 release by default https://review.openstack.org/409071 | 21:47 |
| openstackgerrit | Dirk Mueller proposed openstack/diskimage-builder: Switch to openSUSE Leap 42.2 release by default https://review.openstack.org/409071 | 21:48 |
| openstackgerrit | Merged openstack/diskimage-builder: Don't set the executable bit on dhcp-interface@.service https://review.openstack.org/409863 | 22:56 |
| ianw | greghaynes: mind if i change it to "dib_python_version" in the pkg map, just to make clear it's relationship to DIB_PYTHON_VERSION in 408293 ? | 22:57 |
| openstackgerrit | Paul Belanger proposed openstack/diskimage-builder: Move adduser to debian element from debootstrap https://review.openstack.org/410458 | 23:26 |
| pabelanger | ianw: so, https://review.openstack.org/#/c/318461/ | 23:28 |
| openstackgerrit | Merged openstack/diskimage-builder: Make dib-python use the default python for distro https://review.openstack.org/408288 | 23:28 |
| ianw | pabelanger: uh oh, this seems like a random, weird issue coming | 23:29 |
| pabelanger | ianw: what are your thoughts about either dropping it again or not using pkg-map? Reason being, I want drop pkg-map dependency as it is pulling python into my debootstrap elements | 23:30 |
| pabelanger | trying to see if I can remove the need for python | 23:30 |
| pabelanger | ianw: Or, maybe moving it into the growroot element | 23:30 |
| ianw | haha, greghaynes is trying to see if he can remove the need for bash! | 23:31 |
| ianw | something seems like it's gotta give there | 23:31 |
| pabelanger | wut? bash | 23:31 |
| pabelanger | haha, I was hoping just for it :) | 23:31 |
| ianw | well, moving to python over bash so busybox images can work | 23:31 |
| pabelanger | Ah, I see | 23:32 |
| ianw | i mean, if we had dib-run-parts in python, you'd be kind of screwed right? | 23:32 |
| ianw | unless i guess you uninstall all the python in a cleanup | 23:32 |
| pabelanger | This is just from me poking at the ubuntu-minimal element, minus kernel and base. Am building some docker containers with DIB | 23:33 |
| ianw | (i'm presuming this is for making a docker image) | 23:33 |
| pabelanger | ya, in fact, I'm trying to drop dib-run-parts too | 23:33 |
| pabelanger | but haven't got there yet | 23:33 |
| pabelanger | currently dib-run-parts element has no element-deps, so not sure if dib-python is needed | 23:35 |
| ianw | no, currently dib-run-parts is in bash | 23:35 |
| ianw | but greghaynes has a wip change to move it to python | 23:35 |
| ianw | for that busybox image build reason | 23:35 |
| pabelanger | right | 23:35 |
| ianw | so you'd basically want it the other way | 23:35 |
| ianw | all in-chroot scripts in shell, no python | 23:36 |
| pabelanger | I suspect this is where the ansible stuff we talked about is going to come into play | 23:36 |
| ianw | that's still going to require python in the chroot, though? | 23:36 |
| pabelanger | run things in ansible outside chroot, then connection chroot | 23:36 |
| pabelanger | ianw: depends on what I'm doing, | 23:36 |
| pabelanger | but eventually ya | 23:36 |
| pabelanger | so, maybe at this point, accept python for now | 23:37 |
| ianw | i feel like python in the chroot is going to be a hard requirement | 23:37 |
| pabelanger | and maybe cleanup in finalize if possible | 23:37 |
| ianw | if anything, we're moving more towards that | 23:37 |
| pabelanger | okay | 23:37 |
| pabelanger | I am fine with that for now | 23:37 |
| ianw | yes, at least for build. if it can be cleaned, that would be fine i guess | 23:37 |
| pabelanger | k | 23:37 |
| pabelanger | let me continue clean up of debootsrap, see if I can get it down a few more packages | 23:38 |
| pabelanger | I'll WIP my ubuntu-container element in a bit, once I get things working | 23:38 |
| pabelanger | okay, have to run out for an errand, once back I'll ping about zookeeper | 23:39 |
| ianw | cool | 23:39 |
| ianw | or we could just move everything to perl, that way nobody would be happy :) | 23:39 |
| openstackgerrit | Merged openstack/diskimage-builder: Put MKFS_OPTS after filesystem type https://review.openstack.org/408408 | 23:58 |
| openstackgerrit | Merged openstack/diskimage-builder: DIB element to support cinder local attach/detach functionality https://review.openstack.org/385880 | 23:58 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!