| *** matsuhashi has joined #savanna | 00:02 | |
| *** matsuhashi has quit IRC | 00:14 | |
| *** matsuhashi has joined #savanna | 00:15 | |
| *** matsuhashi has joined #savanna | 00:15 | |
| *** matsuhashi has quit IRC | 00:29 | |
| *** matsuhashi has joined #savanna | 00:29 | |
| *** matsuhashi has quit IRC | 00:34 | |
| *** matsuhashi has joined #savanna | 01:11 | |
| *** nosnos has joined #savanna | 01:24 | |
| *** matsuhashi has quit IRC | 01:24 | |
| *** matsuhashi has joined #savanna | 01:25 | |
| *** matsuhas_ has joined #savanna | 01:29 | |
| *** matsuhashi has quit IRC | 01:32 | |
| *** matsuhas_ has quit IRC | 01:37 | |
| *** matsuhashi has joined #savanna | 01:38 | |
| *** matsuhas_ has joined #savanna | 01:45 | |
| *** matsuhashi has quit IRC | 01:48 | |
| *** alazarev has joined #savanna | 02:17 | |
| *** alazarev has quit IRC | 02:28 | |
| *** alazarev has joined #savanna | 02:30 | |
| *** bmahe has joined #savanna | 02:33 | |
| *** _mattf has joined #savanna | 03:04 | |
| *** aignatov has joined #savanna | 03:05 | |
| *** alazarev has quit IRC | 03:22 | |
| *** alazarev has joined #savanna | 03:23 | |
| *** alazarev has quit IRC | 03:29 | |
| *** coolsvap has joined #savanna | 03:42 | |
| *** matsuhas_ has quit IRC | 03:46 | |
| *** matsuhashi has joined #savanna | 03:47 | |
| *** matsuhashi has quit IRC | 03:51 | |
| *** matsuhashi has joined #savanna | 04:20 | |
| *** matsuhashi has quit IRC | 04:42 | |
| *** matsuhashi has joined #savanna | 04:42 | |
| *** aignatov has quit IRC | 04:48 | |
| *** IlyaE has joined #savanna | 04:49 | |
| *** IlyaE has quit IRC | 04:59 | |
| *** IlyaE has joined #savanna | 05:03 | |
| *** IlyaE has quit IRC | 05:08 | |
| *** IlyaE has joined #savanna | 05:10 | |
| *** alazarev has joined #savanna | 05:32 | |
| *** nosnos_ has joined #savanna | 05:36 | |
| *** nosnos has quit IRC | 05:36 | |
| *** DennyZhang has joined #savanna | 05:40 | |
| *** IlyaE has quit IRC | 05:52 | |
| *** SergeyLukjanov has joined #savanna | 05:54 | |
| *** IlyaE has joined #savanna | 05:55 | |
| *** nadya has joined #savanna | 05:58 | |
| *** nadya is now known as Guest37625 | 05:58 | |
| *** Guest37625 has quit IRC | 06:00 | |
| *** DennyZhang has quit IRC | 06:04 | |
| *** alazarev has quit IRC | 06:07 | |
| *** alazarev has joined #savanna | 06:08 | |
| *** IlyaE has quit IRC | 06:11 | |
| *** IlyaE has joined #savanna | 06:13 | |
| *** alazarev has quit IRC | 06:26 | |
| *** alazarev has joined #savanna | 06:40 | |
| *** SergeyLukjanov has quit IRC | 06:41 | |
| *** bogdando has joined #savanna | 06:52 | |
| *** matsuhashi has quit IRC | 06:54 | |
| *** matsuhashi has joined #savanna | 06:56 | |
| *** nosnos_ has quit IRC | 06:56 | |
| *** nosnos has joined #savanna | 06:56 | |
| *** IlyaE has quit IRC | 07:01 | |
| *** bmahe has quit IRC | 07:11 | |
| *** akuznetsov has quit IRC | 07:23 | |
| *** IlyaE has joined #savanna | 07:39 | |
| *** DinaBelova has joined #savanna | 07:42 | |
| *** DinaBelova has quit IRC | 07:46 | |
| *** DinaBelova has joined #savanna | 07:51 | |
| *** nadya_ has joined #savanna | 07:53 | |
| *** SergeyLukjanov has joined #savanna | 08:02 | |
| *** nadya_ has quit IRC | 08:06 | |
| *** alazarev has quit IRC | 08:12 | |
| *** IlyaE has quit IRC | 08:12 | |
| *** matsuhashi has quit IRC | 08:13 | |
| *** matsuhashi has joined #savanna | 08:14 | |
| *** matsuhashi has quit IRC | 08:15 | |
| *** matsuhashi has joined #savanna | 08:15 | |
| *** akuznetsov has joined #savanna | 08:17 | |
| *** aignatov has joined #savanna | 08:45 | |
| *** bogdando has quit IRC | 08:48 | |
| *** bogdando has joined #savanna | 08:52 | |
| *** nosnos has quit IRC | 08:59 | |
| *** nosnos has joined #savanna | 08:59 | |
| *** nosnos has quit IRC | 09:03 | |
| *** nprivalova has joined #savanna | 09:36 | |
| *** SergeyLukjanov has quit IRC | 09:51 | |
| *** SergeyLukjanov has joined #savanna | 09:54 | |
| *** matsuhashi has quit IRC | 09:55 | |
| *** matsuhashi has joined #savanna | 09:56 | |
| *** matsuhashi has quit IRC | 10:01 | |
| *** matsuhashi has joined #savanna | 10:05 | |
| *** aignatov has quit IRC | 10:14 | |
| *** akuznetsov has quit IRC | 10:22 | |
| openstackgerrit | Sergey Lukjanov proposed a change to openstack/savanna: Replace unicode() with six.text_type() https://review.openstack.org/56899 | 10:41 |
|---|---|---|
| *** matsuhashi has quit IRC | 10:47 | |
| *** matsuhashi has joined #savanna | 10:47 | |
| *** matsuhashi has quit IRC | 10:52 | |
| *** matsuhashi has joined #savanna | 10:54 | |
| *** akuznetsov has joined #savanna | 10:56 | |
| *** aignatov has joined #savanna | 11:02 | |
| *** aignatov_ has joined #savanna | 11:06 | |
| *** aignatov has quit IRC | 11:07 | |
| *** aignatov_ is now known as aignatov | 11:07 | |
| *** matsuhashi has quit IRC | 11:13 | |
| *** matsuhashi has joined #savanna | 11:14 | |
| *** matsuhashi has quit IRC | 11:19 | |
| *** DinaBelova has quit IRC | 11:19 | |
| *** SergeyLukjanov has quit IRC | 11:24 | |
| *** SergeyLukjanov has joined #savanna | 11:27 | |
| *** matsuhashi has joined #savanna | 11:32 | |
| *** DinaBelova has joined #savanna | 11:36 | |
| *** akuznetsov has quit IRC | 11:37 | |
| *** akuznetsov has joined #savanna | 11:52 | |
| *** DinaBelova has quit IRC | 12:01 | |
| *** coolsvap has quit IRC | 12:09 | |
| *** nprivalova has quit IRC | 12:29 | |
| *** IlyaE has joined #savanna | 12:30 | |
| *** nprivalova has joined #savanna | 12:33 | |
| *** IlyaE has quit IRC | 12:34 | |
| *** IlyaE has joined #savanna | 12:43 | |
| *** IlyaE has quit IRC | 12:47 | |
| *** SergeyLukjanov is now known as _SergeyLukjanov | 12:53 | |
| *** _SergeyLukjanov has quit IRC | 12:53 | |
| *** coolsvap has joined #savanna | 12:55 | |
| *** matsuhashi has quit IRC | 13:05 | |
| *** matsuhashi has joined #savanna | 13:05 | |
| *** _crobertsrh is now known as crobertsrh | 13:13 | |
| *** SergeyLukjanov has joined #savanna | 13:44 | |
| *** IlyaE has joined #savanna | 13:48 | |
| *** matsuhashi has quit IRC | 13:56 | |
| *** matsuhashi has joined #savanna | 13:57 | |
| *** sballe has quit IRC | 13:57 | |
| *** matsuhas_ has joined #savanna | 13:58 | |
| *** matsuhashi has quit IRC | 13:59 | |
| *** _mattf is now known as mattf | 14:13 | |
| *** aignatov has quit IRC | 14:17 | |
| *** tmckay has joined #savanna | 14:19 | |
| *** SergeyLukjanov is now known as _SergeyLukjanov | 14:32 | |
| *** _SergeyLukjanov has quit IRC | 14:32 | |
| *** nprivalova has quit IRC | 14:32 | |
| *** nprivalova has joined #savanna | 14:36 | |
| *** matsuhas_ has quit IRC | 14:57 | |
| *** matsuhashi has joined #savanna | 14:57 | |
| openstackgerrit | A change was merged to openstack/savanna: Fix style errors and upgrade hacking https://review.openstack.org/56405 | 14:58 |
| openstackgerrit | A change was merged to openstack/savanna: Remove check already released in hacking 0.8.0 https://review.openstack.org/56568 | 14:58 |
| *** SergeyLukjanov has joined #savanna | 15:02 | |
| *** matsuhashi has quit IRC | 15:04 | |
| *** matsuhashi has joined #savanna | 15:04 | |
| *** SergeyLukjanov has quit IRC | 15:04 | |
| openstackgerrit | A change was merged to openstack/savanna: Replace unicode() with six.text_type() https://review.openstack.org/56899 | 15:08 |
| *** matsuhashi has quit IRC | 15:08 | |
| *** SergeyLukjanov has joined #savanna | 15:10 | |
| *** aignatov has joined #savanna | 15:16 | |
| *** nprivalova is now known as nadya_ | 15:18 | |
| *** alazarev has joined #savanna | 15:22 | |
| openstackgerrit | Sergey Lukjanov proposed a change to openstack/savanna: Hacking contains all needed requirements https://review.openstack.org/56531 | 15:27 |
| openstackgerrit | A change was merged to openstack/savanna: Hacking contains all needed requirements https://review.openstack.org/56531 | 15:42 |
| openstackgerrit | Vadim Rovachev proposed a change to openstack/savanna-dashboard: Added EDP tests https://review.openstack.org/56235 | 15:55 |
| *** aignatov has quit IRC | 16:09 | |
| openstackgerrit | Sergey Reshetnyak proposed a change to openstack/savanna: Fix getting hidden vanilla plugin parameters https://review.openstack.org/56975 | 16:13 |
| *** SergeyLukjanov has quit IRC | 16:13 | |
| openstackgerrit | Vadim Rovachev proposed a change to openstack/savanna-dashboard: Added EDP tests https://review.openstack.org/56235 | 16:20 |
| *** alazarev has quit IRC | 16:23 | |
| *** nadya_ has quit IRC | 16:28 | |
| *** alazarev has joined #savanna | 16:30 | |
| *** alazarev has quit IRC | 16:30 | |
| *** rnirmal has joined #savanna | 16:31 | |
| *** alazarev has joined #savanna | 16:31 | |
| openstackgerrit | Vadim Rovachev proposed a change to openstack/savanna-dashboard: Added EDP tests https://review.openstack.org/56235 | 16:31 |
| *** alazarev has joined #savanna | 16:52 | |
| *** tmckay has quit IRC | 16:59 | |
| *** SergeyLukjanov has joined #savanna | 17:01 | |
| *** akuznetsov has quit IRC | 17:04 | |
| *** aignatov has joined #savanna | 17:04 | |
| *** nadya_ has joined #savanna | 17:32 | |
| *** aignatov has quit IRC | 17:48 | |
| *** aignatov has joined #savanna | 17:57 | |
| *** tmckay has joined #savanna | 18:01 | |
| *** nadya_ has quit IRC | 18:06 | |
| *** nadya_ has joined #savanna | 18:07 | |
| *** coolsvap has quit IRC | 18:28 | |
| *** akuznetsov has joined #savanna | 18:30 | |
| *** rnirmal has quit IRC | 18:58 | |
| *** IlyaE has quit IRC | 19:07 | |
| *** nadya_ has quit IRC | 19:14 | |
| *** akuznetsov has quit IRC | 19:15 | |
| *** nadya_ has joined #savanna | 19:18 | |
| *** jcooley_ has joined #savanna | 19:41 | |
| *** nadya_ has quit IRC | 19:43 | |
| *** IlyaE has joined #savanna | 19:48 | |
| *** aignatov has quit IRC | 19:48 | |
| *** SergeyLukjanov has quit IRC | 19:52 | |
| *** IlyaE has quit IRC | 19:53 | |
| *** dc has joined #savanna | 19:53 | |
| openstackgerrit | Jonathan Maron proposed a change to openstack/savanna: Enable EDP with HDP plugin https://review.openstack.org/56274 | 19:53 |
| *** IlyaE has joined #savanna | 19:56 | |
| openstackgerrit | Chad Roberts proposed a change to openstack/savanna-dashboard: Adding mains and libs to job details view https://review.openstack.org/57030 | 19:56 |
| dc | Is this the proper place to discuss Savanna's swift integration with hadoop? | 19:56 |
| crobertsrh | dc: Yes, this would be a fine place. | 19:57 |
| dc | crobertsrh: thank you. With the whole swift & hadoop integration, was swift meant to replace hdfs or work along side it? | 19:58 |
| crobertsrh | I think it's meant to work alongside it, but I'm far from the definitive expert on that. Hopefully, someone else can chime in here. | 20:00 |
| mattf | i wonder if we can get stevel over here too | 20:00 |
| mattf | it's kinda late in the day for him though | 20:00 |
| mattf | dc, the idea is to use swift as the source of the MR input and destination for the MR output | 20:01 |
| mattf | dc, it's similar functionality to what you'll find in AWS's EC2 + S3 integration | 20:01 |
| dc | mattf: All, please excuse my ignorance as this is new to me. If Swift and Hadoop are used together, what would be a use case of when an object would be stored in swift as compared to the hdfs? | 20:03 |
| *** dc_ has joined #savanna | 20:05 | |
| dc_ | sorry..I lost my connection to the cat so I will repost my question | 20:05 |
| mattf | dc_, still there? | 20:05 |
| *** jcooley_ has quit IRC | 20:05 | |
| mattf | we're mostly providing the tools to build up interesting uses cases. we've currently provided an EMR-like workflow that allows you to store input&output in swift | 20:06 |
| dc_ | mattf: Please pardon my ignorance...if Swift and hdfs are to be used together, what is the use case of when to store an object in Swift instead of the hdfs and vice versa? | 20:06 |
| mattf | the next version should support in/out in hdfs | 20:06 |
| mattf | the main case i see perssuring this is intermediate data in hdfs. i.e. MR-job0 -> MR-job1 etc where the input to job0 comes from swift and the output on job1 may go to swift. | 20:07 |
| *** dc has quit IRC | 20:07 | |
| mattf | dc_, ^^ | 20:09 |
| dc_ | mattf: It took me some time but I think I got my hadoop and swift integrated. I ran the command on my hadoop single node cluster : hadoop distcp hdfs://localhost:53410/app/hadoop/tmp/core-site.xml swift://opencloudservices.swift/temp | 20:09 |
| dc_ | and I saw the core-site.xml on my swift cluster | 20:10 |
| mattf | that's promising. are you having issues w/ swift via edp? | 20:10 |
| dc_ | mattf: edp? | 20:11 |
| dc_ | mattf: endpoint? | 20:11 |
| mattf | "elastic data processing" it's the feature we're we implemented elastic mapreduce-like functionality | 20:11 |
| mattf | dc_, what are you trying to do? | 20:11 |
| dc_ | mattf: I just was trying to see if I could get an object copied from hdfs into the swift cluster. | 20:12 |
| mattf | glad it worked then! | 20:13 |
| dc_ | mattf: I'm hoping that once the functionality is there that people can program around placing objects in swift. I hope that makes sense. I'm just a sysadmin :( | 20:13 |
| *** nadya_ has joined #savanna | 20:13 | |
| dc_ | mattf: If you don't mind me asking this...what are the benefits of using swift instead of hdfs? Is it cheaper? | 20:14 |
| mattf | since the swift plugin uses the standard distributedfilesystem api in hadoop, it should be transparent to folks. they just have to use the correct uri. | 20:14 |
| mattf | dc_, if you have a persistent cluster w/ hdfs the benefit may be small for you. you may store relatively static datasets in swift for sharing across multiple hadoop clusters | 20:15 |
| dc_ | mattf: what do you mean exactly by "persistent cluster with hdfs"? | 20:17 |
| *** rnirmal has joined #savanna | 20:18 | |
| mattf | dc_, say you start a cluster w/ savanna and let multiple people run jobs on it. alternatively, and which is typical now, you have a cluster started for each job you want to run. | 20:18 |
| dc_ | mattf: Thanks alot..Have you worked on the Savanna project? | 20:19 |
| mattf | dc_, i'm definitely helping out here, yes | 20:20 |
| dc_ | mattf: I can tell..You definitely know your stuff. | 20:21 |
| dc_ | Thanks again | 20:21 |
| *** alazarev has quit IRC | 20:22 | |
| *** nadya_ has quit IRC | 20:22 | |
| mattf | dc_, my pleasure. lots of folks here to help. | 20:22 |
| *** dc_ has quit IRC | 20:26 | |
| *** IlyaE has quit IRC | 20:27 | |
| *** alazarev has joined #savanna | 20:32 | |
| *** alazarev has quit IRC | 20:36 | |
| *** IlyaE has joined #savanna | 20:42 | |
| *** IlyaE has quit IRC | 20:42 | |
| *** IlyaE has joined #savanna | 20:44 | |
| *** jcooley_ has joined #savanna | 20:45 | |
| *** jcooley_ has quit IRC | 20:46 | |
| *** alazarev has joined #savanna | 20:58 | |
| *** IlyaE has quit IRC | 21:07 | |
| *** IlyaE has joined #savanna | 21:10 | |
| *** IlyaE has quit IRC | 21:14 | |
| *** aignatov has joined #savanna | 21:24 | |
| *** aignatov has quit IRC | 21:25 | |
| *** jcooley_ has joined #savanna | 21:43 | |
| *** crobertsrh is now known as _crobertsrh | 21:43 | |
| *** mattf is now known as _mattf | 22:16 | |
| *** jcooley_ has quit IRC | 22:38 | |
| *** alazarev has quit IRC | 23:40 | |
| *** alazarev has joined #savanna | 23:43 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!