401–410 of 466 posts
Docker lets me iterate on system configuration faster than ever, and that means learning the details and quirks of certain software faster. Then again, I usually don't use prebuilt VMs and containers, but have to prepare them for people who don't want to pay for good sysadmins..
Earlier quoted context omitted.
Hadoop is insane. The elephant is fitting. Is it really the best choice, or has someone done something cleaner in golang or c++11?
Apache Spark is a good replacement for Hadoop now. It's written in Scala.
Earlier quoted context omitted.
There are many more depressing examples of this at http://curlpipesh.tumblr.com
Funny tumblr but makes me care-confused. I understand that curl pipe sh could have security problems but I also don't see it as that much different than the "normal" and "ok" way of doing things. I would consider something like the below pretty normal. wget https://whatever.io/latest.tgz tar xzf latest.tgz cd whatever-stable ./configure && make sudo make install Because of familiarity, we aren't going to be too worri…
* Using checkinstall to create a local deb/rpm which can be easily installed/removed later instead of "make install". * What if installer.sh says "rm -rf /tmp/PACKAGE-build" and the connection is interrupted just after the first "/", you now have "rm -rf /". Oops. * configure will tell you what files it needs, and apt-file will tell you want dependencies to install. * I know what make install does. I know make. Who wrote installer.sh? Do they know anything about writing good software? Steam wiped out home directories, who knows what these people do
Earlier quoted context omitted.
>> No one can tell you how to build hadoop? Well, figure it out. I get the impression that several people working on debian couldn't work this one out!
I think most people who use debian would tend to install things using debian packages, which in this case usually means adding cloudera to your apt sources list and using apt-get. It is a pretty straightforward process: http://www.cloudera.com/content/cloudera/en/documentation/cd...
Earlier quoted context omitted.
Nah, man pages are usually completely useless. I use man when I remember exactly what I want to do and just aren't sure if the flag was -f or -F. For everything else there's google.
Can you give an example of what you might turn to google for (and what you'd search for) that is more productive than checking a manpage/help output?
Google query: git display file at revision. Immediate answer (without even having to click any links, it's in the result description): `git show revision:file`
Total time: 5 seconds
Trying to reproduce with man and help:
man git
start scrolling down
notice git-show (show various types of objects); sounds like a likely candidate
git show
git show -h
usage: git log [] [..] [[--] ...]
or: git show [options] ...
man git show
man git-show
man git-rev-parse
Give up. Google it.
Earlier quoted context omitted.
Apache Spark is a good replacement for Hadoop now. It's written in Scala.
Spark is a good replacement for MapReduce. MapReduce != Hadoop.
"without even using any of the HBaseGiraphFlumeCrunchPigHiveMahoutSolrSparkElasticsearch (or any other of the Apache chaos) mess yet."
I agree that many of these convenient setups are embarrassingly sloppy, but it's the sysadmin's responsibility to insist on production deployments being far more rigorous. No one can tell you how to build hadoop? Well, figure it out. Random Docker containers being downloaded? Use a local Docker repo with vetted containers and Dockerfiles only. I don't even allow vendor installers to run on my production systems. My e…
> it's the sysadmin's responsibility to insist on production deployments What decade are you from? No startups are hiring sysadmins to do any kind of work anymore. They're hiring "dev-ops" people, which seems to mean "Amateur $popularLanguage developer that deployed on AWS this one time." That's the whole problem with the dev-ops ecosystem. None of these dev-ops people seem to have any ops experience.
Thanks for painting all of us that do "devops" with a wide brush. If you're a dev shall we enumerate all of the XSS and SQL injection holes you've added to products over your career?
Earlier quoted context omitted.
> If you are trying to be an old-school sysadmin that expects to console or SSH in and run 'yum upgrade' or 'apt-get upgrade' your containers then you are doing containers wrong... A container is a chroot environment for running a service. Basically. It's perfectly possible to 'yum upgrade' or 'apt-get upgrade' them. I think what you're trying to say is this is a bad idea because containers are not supposed to be man…
I corrected myself for the other person to mean '[..] upgrade your docker containers'. I heavily use LXC containers (and had used openvz and vserver before that) and treat them as individual servers. All your points about bad sysadmin practices are OS & container agnostic - they can happen on any platform, don't drag Docker into it. Sure there is a culture of 'docker run somebinaryimage [..]' but those people are the…
The image (and thus container) are the same. Trying to rebuild it from scratch leaves it not the same, typically because the way people put together Dockerfiles and build images does not follow a standard. And it has to do with the container culture.
Let's take a Dockerfile for the CentOS 7 version of nginx (https://github.com/CentOS/CentOS-Dockerfiles/blob/master/ngi...)
FROM centos:centos7
MAINTAINER The CentOS Project
RUN yum -y update; yum clean all
RUN yum -y install epel-release tar ; yum clean all
RUN yum -y install nginx ; yum clean all
ADD nginx.conf /etc/nginx/nginx.conf
RUN echo "daemon off;" >> /etc/nginx/nginx.conf
RUN curl https://git.centos.org/sources/httpd/c7/acf5cccf4afaecf3afeb18c50ae59fd5c6504910 \
| tar -xz -C /usr/share/nginx/html \
--strip-components=1
RUN sed -i -e 's/Apache/nginx/g' -e '/apache_pb.gif/d' \
/usr/share/nginx/html/index.html
EXPOSE 80
CMD [ "/usr/sbin/nginx" ]System administration is as important as ever. Docker and other containers just simplify system administration across many different machines. The standard Unix user land tools are excellent and very flexible, but they are fucking god awful at configuration management. Docker solves the problem of "how do I make sure I have the same versions and configurations of everything on all 500 of my compute nodes without havi…
Many places are rolling this type of sysadmin work up into DevOps. This scares graybeard sysadmins, because they see DevOps automating them out of a job. Nope, not really. just wait till you move to a new job, and you inherit a docker/rockit/etc system. You need to patch openssl/glibc/etc however, half the containers are built with an old build system that's been replaced. You've got 15 containers based on fedora20 w…
>An example, Say you want 10 VMs of x size. If you have a coheesive system, you could email/phone/talk to a guy and you'll have some machines. If your provisioning team had thought ahead they'll have made an API that spins up machines, ties them to your accounting code, and configured them to your environment. That's not technology, that just good practice.
Or you know, I have a fleet of ten thousand machines that I can schedule whatever docker containers I want into.
Your nonsense about software being out of date and EOL is absolutely no different than a scenario without docker. At least docker gives you the change to have a declarative way of reconstructing the image.
I can't honestly believe that was what you went with to try to make containers look bad. You're just showing your age is all.