Welcome back, my nascent hackers! Although my favorite TV show, Mr. Robot, had completed its first season already, I have not completed demonstrating the hacks that Elliot used in the show. (By the way, I can't wait for season 2!) In several episodes, Elliot was seen sending secure emails. As a seasoned and savvy hacker, he would never use Gmail, Hotmail, or Yahoo to send confidential messages. Instead, Elliot chose the most secure email system available to non-military and non-spy people like you and me: ProtonMail. In a world where Google and the NSA inspect every email, a truly secure... more
Our Xen Project Test Days help insure that upcoming releases are ready for production, beyond what our automated testing through our Test Lab can accomplish. It is particularly important that our users test out the upcoming release in their own environment. We rely on your functional testing of features, stress-testing, edge case testing, and performance testing to prove that the code is ready for consumption. And this is your opportunity to verify that the new code will continue to work well in your particular situation.
Continuing our current release cycle, the Test Day for Xen Project 4.6 RC4 has been set for Thursday, October 1, 2015.
This may be the final RC before release, so the time to test the software is now!
Additrional information about Test Days can be found here:
Join us on Tuesday in #xentest on Freenode IRC!
Test a Release Candidate! Help others, get help! And have fun!
(Noun)
A reciprocal, parallel or complementary relationship between two or more comparable objects
What about that? Well, you'll see, correlations are a very important thing to help you every day.
We'll show you how we leverage the XenServer API to give you hints about what's happening in your virtualized infrastructure.
TL;DR? Okay, just a quick preview:
Let's explore what we already have and what we need.
Having stats and metrics on your infrastructure is a very common thing in IT now. For example, Xen Orchestra already offers you instant statistics in a VM or a host view:
We also introduced the concept of heatmap to notice the differences or "hot spots" on the last 7 days:
One thing is missing: what about displaying the same metric for various hosts or VM? You should be able exploit this possibility to correlate through the time any pike or remarkable value.
But there is multiple challenges:
Let's tackle the first problem: limited vertical space. A classical line graph or area chart will be "flattened" when displayed in a small vertical space.
If you compress a 120-pixel tall area chart to 30 pixels, you lose 75% of the resolution and it becomes harder to see small changes
Source: https://square.github.io/cubism/
That's why the concept of horizon charts is really good: larger values are "overplotted" in successively darker colors.
Example:
Okay, done! Next :)
What about the "two scales" problem? When you compare multiple objects, you can also have some of them with far higher values than others, thus hiding small correlations. But also sometimes, we need a "synchronized" or consistent scale. How to choose between those? The answer is: let the user make the choice!
Let's display the network interface n°1 upload for 14x VMs used in production, with a "synchronized" scale (consistent):
Okay, we can note:
Let's deactivate the consistent scale on this very graph and see the result:
Do you notice it? This spike on all VMs is now visible because each VM has its own scale. For those wondering: it's a Salt Stack pkg.upgrade command.
And with hosts? Thanks to our data consolidation, we can have an average on all CPUs of your servers and compare them:
Yeah, "Centaurus" is working more than "Andromeda", that's obvious ;)
There is some room from improvement:
This feature will be available... tomorrow in the next release of Xen Orchestra (4.6). Stay tuned!
As promised, here it comes XO 4.6! The complete changelog is here.
Mainly, this release is the result of a major bug hunt plus two new cool features: tags management and event correlation visualization (picture below):
You can now use XenServer tags and that's totally compatible with XenCenter (you will keep all your existing tags and continue to use them with both XO and XC).
Now, a real example: imagine you want to add a tag on a SR to find it easily after, thanks to our search bar. That's very simple! See:
This will be improved for gathering resources together in folders for the next interface redesign.
Here is the new datavisualization for your infrastructure! This time, it's about how to detect event correlation on multiple hosts or VMs.
In few words, it helps to display/compare similar metrics between hosts or VMs. It's very helpful to detect or diagnose problematic events. Read how it will help you in our blog post.
You can now modify a user/group ACL on the fly:
A user created internally (ie: not with a external authentication provided) can change his password, just by clicking on his username on the top right of the screen.
You developers/clients/whatever can now authenticate through their Google account. The configuration is pretty self explanatory (well, if you know Google configuration for SSO):
plugins: auth-google: clientID: xxxxx.apps.googleusercontent.com clientSecret: -HTDb8I4jXiLRMaRL15qCffQ callbackURL: 'http://xo.company.net/signin/google/callback' 
A Console is now automatically restarted even if the connection is ended on XenServer side: it means the console will be displayed again after a reboot.
Now, if you want to modify the PV args, you could do it during the VM creation process:
You can check them even after the VM is created in the VM view:
They are also editable anytime after the VM is created.
When you export a VM through the web UI, you got a "export.xva" file previously. Not very handy to remember which one your downloaded, or you needed to rename it after the download.
That's why this simple fix allows to export a XVA file with the name of the VM, e.g "webserver1.xva".
When you are in a SR view, you can see a lot of VDI displayed. How to find something I want? Now with the filter, it's far easier:
It (live) filters on the VDI name, description and attached VM name.
XOA 4.6 is now running with Node 4.1! Also, it appears to fix some memory leaks.
4.7 will be release in another 15 days. But we'll just have at least 2 new guys working with us: they will need a little bit of time to discover XO before fixing issues and creating new features ;)
We're really excited about the programming we have planned for the DevOps Enterprise Summit 2015. Here are some things you may have expected:
During last year's event, we established that many large organizations were already on the path to adopting DevOps practices, and our speakers shared their struggles from the stage.
After the event, we wanted to learn more about what the biggest roadblocks were for our attendees in their own companies, so what did we do?
After analyzing the data from the survey, we noticed that these important questions came up again and again…
Top results from the survey sent to attendees:
- How do we adopt DevOps practices in legacy environments?
- How do we successfully lead large transformations in technology organizations?
- What are the best approaches for organizational design, roles and responsibilities?
- How do we integrate security and compliance into our DevOps efforts?
- What metrics work best for improving performance in DevOps initiatives?
To help the DevOps Enterprise community overcome these obstacles, this year we have gathered the top experts, case studies, and practitioner-led research to answer these five important questions.
We believe the answers to these questions will accelerate the adoption and success of DevOps initiatives in enterprise organizations around the world.
1. For the tough challenge of legacy environments…
You'll hear experience reports from century-old companies Sherwin-Williams and Western Union on how they have accelerated their DevOps practice and IBM's Rosalind Radcliffe will talk about how to create test automation in mainframe applications.
2. For leading large scale transformations…
MIT's Dr. Steven Spear will discuss the cultures and management methods in high-velocity organizations such as Toyota, Alcoa, and the U.S. Naval Reactor program, and Paula Thrasher will talk her work mobilizing IT transformations in the face of extreme bureaucracies in more than over 16 different federal government agencies.
3. For enabling organizational design…
Ralph Loura, CIO for HP's Enterprise Group, will talk about nurturing DevOps capabilities and spreading those practices through a mammoth technology organization, and Jason Cox, Director of Systems Engineering for The Walt Disney Company will share how he organizes, staffs and funds embedded Operations engineer across the Disney enterprise.
4. For security and compliance best practices…
Both Ed Bellis, former CISO of Orbitz, and Bill Shinn, principal solutions architect at Amazon Web Services, will talk about how to overcome security and compliance objections, and integrate those practices into publicly traded companies, and some of the most regulated and security-stringent organizations around.
5. For using better metrics…
Julia Wester of LeanKit and Troy Magennis of Focused Objectives will talk about the theory and practice of making team data visible in ways that lead to improvement action and how to avoid the pitfalls and traps of managing by numbers.
Target's Heather Mickman and Ross Clanton on how APIs enable their developer productivity and how to spread DevOps practice into the whole organization.
Nordstrom's Courtney Kissler will talk about their company wide goal of reducing lead time by 20%.
AND you'll hear about the answers to other burning questions like:
The real purpose of the DevOps Enterprise Summit is to create a community of practice and to create a place where practitioners can share what they have learn.
I hope you will join us for another year of this amazing event. —Gene Kim
Register now with the code GENEKIM10 and get 10% off! >>>
The post What's new at DevOps Enterprise Summit 2015 appeared first on IT Revolution.
Manage 2.0.0 is now available from the Chef downloads site.
This release gets a major version bump from 1.21.1 to 2.0.0, with the most major change being that the package is now called "chef-manage" instead of "opscode-manage."
It's a major version bump because of the name change, but you shouldn't find breaking changes or any major overhaul of the web interface.
This release is not currently running on hosted Chef, but is only available for on-premises installation. It will be deployed to hosted Chef soon.
The full change log is shown below.
chef-server-ctl install opscode-manage will install a version prior to 2.0.0. To install the latest version, you must use apt-get install chef-manage.When using Enterprise Linux, chef-server-ctl install opscode-manage will install the latest version of chef-manage.
Ohai Chefs! Time for a shiny new ChefDK release :)
ChefDK 0.8.0 is now available at https://downloads.chef.io/chef-dk/ on OS/X (including OS X 10.11, "El Capitan"), Red Hat Enterprise Linux, Windows, Ubuntu and Debian!
Highlights include:
chef delete-policy, delete-policy-group, clean-policy-cookbooks and clean-policy-revisions. Requires Chef Server 12.2.chef shell-init)!ignore_converge_failure option to not fail the provisioning run of a provisioned node fails chef-client.aws_route_table now accepts an aws_vpc_peering_connection as a route target, see aws_route_table_spec.rb for an example.machine resource to use the AWS SDK V2 instead of V1.bootstrap_options should continue to work, but we now support the full hash of options listed in the AWS documentation except the count attributes.network_interfaces and block_device_mappings than the V1 SDK supported.aws_tags attribute.from_image in the machine_options now correctly works to create an instance from an AMI or machine_image resource.load_balancer with the health_check hash in the load_balancer_options attribute.use_private_ip_for_ssh. Replaced it with transport_address_location which can have an option of :public_ip, :private_ip or :dns.AWS::EC2::Errors::RequestLimitExceeded error – see the section in the README for details.Citrix just announced the availability of Dundee Beta 1. So we are moving toward to a release of this brand new XenServer.
What does it mean in concrete terms? Let's explore it.
It works flawlessly with Xen Orchestra, and more: it will be even better. Why better?
Xen Orchestra connected to a Dundee host, with the host console.
Another good news? Maybe Storage Repositories (SR) metrics via RRDs (I didn't verify it for the Beta 1, I'll keep you posted!)
I experienced some difficulties with previous versions for install XenServer from an USB drive.
Guess what?
dd if=XenServer-6.6.90-install-cd.iso of=/dev/sdd And you're done! This is really great.
XenServer Dundee Beta 1 come with:
Xen 4.6? Hell yeah! What about PVH guest support? I don't know for now, but be sure I'll dig on this topic. Is it enough mature to be in XenServer? We'll see.
From the original article of xenserver.org:
[thanks to cgroups] VM start operations no longer block as before resulting in VM start times being much more equitable. This same optimization can be seen in other VM operations such as when large quantities of VMs are shutdown.
Is it real? Let's try it!
On my small test machine, with a slow HDD SR, I booted a lot of (26x) small Debian VMs:
By the way, just check the start during the boot of all of them:
Let's shutdown them at once! See the stats:
So roughly, Dundee is twice as fast for "shutdown storm" as Creedence!
For sure, I'll run more benchmarks as soon as I can, but really: XenServer Dundee will be a great (the best?) release.
We are very pleased to make the first beta of XenServer Dundee available to the community. As with all pre-release downloads, this can be found on the XenServer Preview page. This release does include some potential commercial features, and if you are an existing Citrix customer you can access those features using the XenServer Tech Preview. It's also important to note that a XenServer host installed from the installer obtained from either source will have identical version number and identical functionality. Application of a Tech Preview license unlocks the potential commercial functionality. So with the "where do I get Dundee beta 1" out of the way, I bet you're all interested in what the cool bits are, and what things might be worth paying attention to. With that in mind, here are some of the important platform differences between XenServer 6.5 SP1 and Dundee beta 1.
The control domain, dom0, has undergone some significant changes. Last year we moved to a 64 bit control domain with a 3.10 kernel as part of our effort to increase overall performance and scalability. That change allowed us to increase VM density to 1000 VMs per host while making some significant gains in both storage and network performance. The dom0 improvements continue, and could have a direct impact on how you manage a XenServer.
dom0 now uses CentOS 7 as it's core operating system, and along with that change is a significant change in how "agents" and some scripts run. CentOS 7 has adopted systemd, and by extension so too has XenServer. This means that shell scripts started at system initialization time will need to change to follow the unit and service definition model specified for systemd.
Certain xapi processes have been isolated into Linux control groups. This allows for greater system stability under extreme resource pressure which has created a considerably more deterministic model for VM operations. The most notable area where this can be observed is under bootstorm conditions. In XenServer 6.5 and prior, starting large numbers of VMs could result in start operations being blocked due to resource contention which could result in some VMs taking significantly longer to start than others. With xapi isolation into cgroups, VM start operations no longer block as before resulting in VM start times being much more equitable. This same optimization can be seen in other VM operations such as when large quantities of VMs are shutdown.
XenServer 6.5 and prior used an older version of Likewise to provide Active Directory. Likewise is now known as Power Broker, and XenServer is using the Power Broker Identity Services to provide authentication for RBAC. This has improved performance, scale and reliability, especially for complex or nested directory structures. Since RBAC is core to delegated management of a XenServer environment, we are particularly interested in feedback on any issues users might have with RBAC in Dundee beta 1.
In XenServer 6.5 and prior, dom0 disk space was limited to 4GB. While this size was sufficient for many configurations, it was limiting for more than a few of you. As a result we've split dom0 disk into three core partitions; system, log and swap. The system partition is now 18GB which should provide sufficient for some time to come. This also means that the overall install space required for XenServer increases from 8GB to 46GB. As you can imagine, given the importance of this major change, we are very interested to learn of any situations where this change prevents XenServer from installing or upgrading properly.
Having flexible storage options is very important to efficient operation of any virtualization solution. To that end, we've added in support for three highly requested storage improvements; thin provisioned block storage, NFSv4 and FCoE.
iSCSI and HBA block storage can now be configured to be thinly provisioned. This is of particular value to those users who provision guest storage with a high water mark expecting that some allocated storage won't be used. With XenServer 6.5 and prior, the storage provider would allocate the entire disk space which could result in a significant reduction in storage utilization which in turn would increase the cost of virtualization. Now block storage repositories can be configured with an initial size and an increment value. Since storage is critical in any virtualization solution, we are very interested in feedback on this functional change.
Fibre Channel over Ethernet is protocol which allows Fibre Channel traffic to travel over standard ethernet networks. XenServer now is able to communicate with FCoE enabled storage solutions, and can be configured at install time to allow boot from SAN with FCoE. If you are using FCoE in your environment, we are very interested in learning both any issues as well as learning what CNA you used during your tests.
Many additional system level improvements have been made for Dundee beta 1, but the following highlight some of the operational improvements which have been made.
XenServer 6.5 and prior required legacy BIOS mode to be enabled on UEFI based servers. With Dundee beta 1, servers with native UEFI mode enabled should now be able to install and run XenServer. If you encounter a server which fails to install or boot XenServer in UEFI mode, please provide server details when reporting the incident.
XenServer can now optionally generate a server status report on a schedule and automatically upload it to Citrix Insight Services (formerly known as TaaS). CIS is a free service which will then perform the analysis and report on any health issues associated with the XenServer installation. This automatic health check is in addition to the manual server status report facility which has been in XenServer for some time.
Application of XenServer patches through XenCenter has become easier. The XenCenter updates wizard has been rewritten to find all patches available on Citrix's support website, rather than ones that have been installed on other servers. This avoids missing updates, and allows automatic clean-up of patches files at the end of the installation.
These platform highlights speak to how significant the engineering effort has been to get us to beta 1. They also overshadow some other arguably core items like a move to the Xen Project Hypervisor 4.6, host support for up to 5TB of host RAM or even Windows guest support for up to 1TB RAM. What they do show is our commitment to the install base and their continued adoption of XenServer at scale. Last year we ran an incredibly successful prerelease program for XenServer Creedence, and its partly through that program that XenServer 6.5 is as solid as it is. We're building on that solid base in the hopes that Dundee will better those accomplishments, and we're once again requesting your help. Download Dundee. Test it in your environment. Push it, and let us know how it goes. Just please be aware that this is prerelease code which shouldn't be placed in production and that we're not guaranteeing you'll ever be able to upgrade from it.
Download location: http://xenserver.org/prerelease
Defect database: https://bugs.xenserver.org
In January of 2015, I had just graduated with a Masters Degree in Particle Physics. Usually after finishing a masters in physics the next step it to pursue a PhD, however, the prospect of another 5-8 years of schooling did not appeal to me. While in college I did robotics research as well as mathematical finance research, and even built a rudimentary algorithmic trading program using just Java and Yahoo Finance, which worked very well. I really enjoyed the algorithmic trading project and it always stayed in the back of my mind as something I would like to pursue further.
After some job searching, I realized that my educational choices did not lead to a clear career path. Although my Masters degree made me very knowledgeable in radiating dipoles, those are not the "practical" skills that I would need to be a competitive job candidate. So I enrolled in a 12 week data science "bootcamp". After the bootcamp, I began looking for an interesting job. I remembered calling Basho Technologies a couple months prior, inquiring about any data science related opportunities, and was told to call back when my bootcamp was completed. So I decided to follow up. After speaking with Matt Digan, an Engineering Manager at Basho, it seemed there was a need for a real-world use case to test the newly created Basho Data Platform (BDP). We discussed possible use cases and realized that I could expand on my previous algorithmic trading program as a summer internship.
For this project I first had to learn: a lot. Before the internship started, I began reading the 'Little Riak Book' which is quite dense. The book's journey through distributed systems thinking and the implementation of Riak prepared me for what's next. Once I had a feel for how Riak KV works, I began to design the project in detail. I knew the program would be best designed when kept quite simple. It begins with an efficient pull all of the raw data from Riak KV using Python in the format of a Spark Resilient Distributed Dataset (RDD). From there, I would format and analyze the data using the parallel nature of Spark, and then write the results back to Riak.
And the project began. Before we get into the code, it's worth noting that this project is only an example and is, of course, not supported in production by Basho. Now, to the process: I analyzed all possible pairs on the New York Stock Exchange (NYSE). Naively, that means roughly 9 millions pairs created from 3000 stocks. I used historical stock time series data from Google Finance to fill the raw data Riak bucket. Then I pulled and sorted all 9 million pairs for data points in the last N days. Now each pair consists of two time series of length N. To generate a signal, I had to run an algorithm on each pair. The algorithm is called a Engle-Granger Test of Cointegration. Engle and Granger won the 2003 Nobel Prize in Economics for formulating this test. The algorithm tests whether or not the differences between two time series are mean reverting. If the differenced time series is mean reverting, then the original two time series are "cointegrated." This property of cointegration can be exploited by noticing that the differenced time series is currently trading far from the mean, which suggests one of the stocks is over-valued and the other under-valued. The trick is designing a reliable "signal" to indicate the optimal time to get into a trade, and another signal for exiting the trade. The beauty of this strategy is that it works in all markets and has very low market exposure. The main risk involved with pair trading is that the cointegrated relationship between two stocks will break. This can be caused by many events such as a scandal, management shuffling, lawsuits, earnings, geopolitics, etc. Basically, no news is good news.
So for my project, I had to create the signal generator. The program would analyze all 9 million pairs, which on a high-end laptop would take almost half a day, spitting out all pairs that were cointegrated and outside a certain deviation from the mean. The program was to do this every night after the markets closed and have a small Amazon Web Services t2.micro "wake up" five t2.mediums, tie together a BDP cluster, run code on the t2.mediums to pull raw data from Google Finance, load N days worth of each stock into a Spark RDD, create all possible pairs, analyze in parallel with Spark, then write back the results to Riak.
Once I began writing the core of the project, Matt introduced me to Python's virtualenv, and at the time I didn't understand the importance of this tool. It turned out that having this separate environment was crucial to testing and deploying the project down the line. I was now able to quarantine the Python libraries I used so that deploying library dependencies to production would be as easy as running pip install requirements.txt. Also, during the initial code writing phase, I was introduced to Vagrant. Vagrant amazed me at how quickly and seamlessly multiple systems can be booted within a single OS. Vagrant became vital to testing my project, and I'm glad I spent the time to really get to know it's functionality. We have many Vagrantfiles for Riak that personify this ease of use.
After about a month into my internship I had coded out the first working version of my project. I wrote a Python library to download data from Google Finance, store it in a useable format in Riak, retrieve the data from Riak, sort and order the data, create all possible pairs of stocks, analyze the pairs, write back the results to Riak. In order to quickly pull data from Riak, I created a Spark RDD with a list of all stocks and would parallelize the RDD so that I could pull data in parallel from Riak. If I were using Java or Scala, the BDP Spark Connector would have been well suited for this task.. Once the data is in the RDD, I wanted to analyze it in parallel as much as possible in order to cut computation time down from 12 hours. Pruning the data was very important, as I only wanted to use "good" stocks (low risk, with sufficient historical data available) in a parallel Spark RDD to create all possible pairs. Once the data is ready for analysis, each pair was mapped to an anonymous function that handled all of the analysis. The results of the analysis either returns a zero if the pair is not ready to be traded or a one accompanied by relevant data if the pair is ready to be traded. Again the results are mapped to an anonymous function that would write the results in parallel back to Riak. While the analytics is relatively straightforward, getting acquainted with Spark and parallel analytics took quite some time as I had no previous experience with Spark.
At this point, I had a working data pipeline in an IPython notebook, but this was not the full project. I still needed to figure out how to make the pipeline fully automated, instead of manually running IPython notebook cells one after the other. I had no previous exposure to automated code deployment, so this seemed like a fun issue to figure out. Matt pointed me towards several technologies that I had never heard of, including: Fabric, Crontab, Ansible, Terraform, Boto and Docker. I looked over all of them to find which fit my needs best. I decided to use Boto, Fabric, and Crontab, and began learning them inside and out. For example, cron takes care of running code at predetermined times in a fairly straightforward manner. Simply put into a file the time, days, and path to the code, and its runs it at that time.
The next part was a bit more difficult. I wanted to have an AWS EC2 t2.micro (a very inexpensive EC2 machine) always on, acting as a cluster launcher. Each morning at one o'clock the launcher would then boot up a five-node BDP cluster of t2.mediums (not so inexpensive, more computing power). These t2.mediums would act as my analytics machine.
I created an Amazon Machine Image with the BDP and my project code already installed, that would act as the base for the five-node cluster. Then, I wrote some code using Boto to boot and shutdown the five-node cluster. At this point IPython Notebook was no longer of any value, so I turned instead to Fabric in order to quickly boot EC2 machines, setup a cluster, and debug my code I wrote a fabfile library to simulate the launcher before putting the project into production. I was able to write a fabric script that would create a working five-node BDP cluster out of thin air on AWS in under three minutes – quite a time saver. Then, I deployed the necessary code to the launcher and cluster using Fabric and waited until the next morning. Each morning I checked the cluster's Riak database for possible trades to see if it worked properly, and it did. So there you have it: a single t2.micro that would launch a pairs trading signal generation program automatically each night at 1 am to a standby, five-node BDP cluster. Running the program on the cluster instead of a single laptop cut run time down from 12 hours to around two hours, and potentially lower if more BDP nodes are added to the cluster.
In the future I hope to add much more to this project. By adding streaming support and machine learning algorithms on top of the generated data and signal, this project could be turned into a fully functioning, cutting edge, trading system. There is a lot of potential left to fulfill, but only so much time in one summer.
My internship with Basho has been an amazing experience. I had the opportunity to create something interesting from scratch, and learned a lot about software development along the way. I couldn't have done it without the guidance and support of Matt Digan and the rest of the Data Platform team. If you'd like to check out my work, you can find it on GitHub at https://github.com/basho-labs/spark-bdp-stockticker-demo
Korrigan
Last week, Red Hat investigated an intrusion on the sites of both the Ceph community project (ceph.com) and Inktank (download.inktank.com), which were hosted on a computer system outside of Red Hat infrastructure.
Ceph.com provided Ceph community versions downloads signed with a Ceph signing key (id 7EBFDD5D17ED316D). Download.inktank.com provided releases of the Red Hat Ceph product for Ubuntu and CentOS operating systems signed with an Inktank signing key (id 5438C7019DCEEEAD). While the investigation into the intrusion is ongoing, our initial focus was on the integrity of the software and distribution channel for both sites.
To date, our investigation has not discovered any compromised code or binaries available for download on these sites. However, we cannot fully rule out the possibility that some compromised code or binaries were available for download at some point in the past. Further, we can no longer trust the integrity of the Ceph signing key, and therefore have created a new signing key (id E84AC2C0460F3994) for verifying downloads. This new key is committed to the ceph.git repository and is also available from https://git.ceph.com/release.asc. All future release git tags will be signed with this new key.
This intrusion did not affect other Ceph sites such as download.ceph.com (which contained some Ceph downloads) or git.ceph.com (which mirrors various source repositories), and is not known to have affected any other Ceph community infrastructure. There is no evidence that build systems or the Ceph github source repository were compromised.
New hosts for ceph.com and download.ceph.com have been created and the sites have been rebuilt. All content available on download.ceph.com has been verified, and all ceph.com URLs for package locations now redirect there. There is still some content missing from download.ceph.com that will appear later today: source tarballs will be regenerated from git, and older release packages are being resigned with the new release key.
The download.inktank.com host has been retired and affected Red Hat customers have been notified, further information is available at https://securityblog.redhat.com/2015/09/17/.
Users of Ceph packages should take action as a precautionary measure to download the newly-signed versions. Please see the instructions below.
The Ceph community would like to thank Kai Fabian for initially alerting us to this issue.
The following steps should be performed on all nodes with Ceph software installed.
Replace APT keys (Debian, Ubuntu)
sudo apt-key del 17ED316D curl https://git.ceph.com/release.asc | sudo apt-key add - sudo apt-get update
Replace RPM keys (Fedora, CentOS, SUSE, etc.)
sudo rpm -e --allmatches gpg-pubkey-17ed316d-4fb96ee8 sudo rpm --import 'https://git.ceph.com/release.asc'
Reinstalling packages (Fedora, CentOS, SUSE, etc.)
sudo yum clean metadata sudo yum reinstall -y $(repoquery --disablerepo=* --enablerepo=ceph --queryformat='%{NAME}' list '*') This is a guest blog from HP's Phil Prasek and Gunjan Kamle detailing the integration between Chef and HP OneView for provisioning bare metal resources.
What if you could bring the simplicity of public cloud infrastructure as code to your private data center? Great news – now you can with the new Chef Provisioning Driver for HP OneView. You can define a single source of truth with Chef recipes that include bare-metal machine configuration and deploy them as part of an automated continuous integration and delivery process.
Here's how it works: HP OneView provides a clean REST API and AMQP message bus for complete control and visibility of your bare-metal infrastructure, using the same semantics you expect from the public cloud. REST APIs are provided for bare-metal server, storage, networking, and OS deployment. Bare-metal machine templates and profiles simplify management at scale and focus on provisioning up to the OS, which makes it a perfect fit for Chef provisioning to layer on top.
In this short video, Gunjan and I give an overview of the solution, a demo, and a code walkthrough of the Chef Provisioning Driver for HP OneView, including the REST API calls.
Five steps to get you started quickly
Step 1: Configure your HP OneView server templates
Log in to your HP OneView appliance and create server profile templates, which are used to define the prototypical configuration for machine instances with consistent compute, network, and storage configurations.
Step 2: Configure OS build plans (OS deployment appliance)
The HP OneView deployment appliance, Insight Control server provisioning (ICsp), provides many pre-packaged build plans. We will need to add a small Chef-related script to an existing build plan. Login to ICsp, click on OS Build plan to edit it, and add an additional Shell script build step located on GitHub.
Step 3: Create and upload a cookbook to your Chef server
Download and extract the Chef Provisioning driver for HP OneView on your workstation. Locate the knife.rb.example file under the examples.chef folder. Copy it and rename your copy knife.rb. Then edit it to include your Chef, HP Oneview and ICsp authorization details, along with the required server provisioning attributes such as IP addresses, DNS, gateways, and so forth.
Chef recipes provide the customization of the servers you wish to provision. To use our example cookbook to install a simple web application, follow the instructions on driver usage example in our GitHub repository README. You can also create your own cookbook or use an existing cookbook on the Chef Supermarket to configure your new server.
Upload your chef cookbook using a simple knife upload command:
Example: knife cookbook upload 'YourChefRecipe'
Step 4: Start the provisioning
You now have everything in place, and the provisioning can begin. Chef will use the Knife configuration file and provisioning recipe in conjunction with the HP OneView driver to provision our server. Chef will go through the entire build process—from bare-metal hardware to the installation and configuration of the application stack to continual management with Chef.
Open a command window and switch to the directory containing your new Chef recipe. Then enter the following commands to (1) download the required Ruby gems, and (2) execute the recipe. Fill in the appropriate path to your recipe in the second command.
$ bundle install
$ bundle exec chef-client -z pathtoprovisioning_recipe.rb
Chef will immediately begin running your recipe.
Step 5: Sit back and relax while your servers are provisioned
You can see the servers being configured on the HP OneView web interface and if desired, you can open consoles to look at the progress.
Chef will bootstrap the new server, which includes downloading the Chef client software and registering the new server with the Chef server. It will then start a Chef-client run on the new server, which will allow it to pull the necessary cookbooks from the Chef server and apply your recipes.When finished, your server is provisioned and running the complete application stack you defined. You can now continue managing and configuring it using Chef.
Reach out to the HP Datacenter Care-Infrastructure Automation (DC-IA) Center of Excellence (CoE) with questions on the Chef Provisioning Driver for HP OneView and the integration we can provide as part of HP's DC-IA service.
To learn more:
Phil Prasek is Chief Technologist, HP OneView at Hewlett-Packard. Gunjan Kamle is the technical lead for the HP Datacenter Care-Infrastructure Automation Center of Excellence (CoE).
We've just released Chef 12.4.2, a minor update with some packaging-related fixes.
gem install and bundle install bugfix: rubygems and bundler have both been upgraded, and will now correctly pull in the latest universal version of a platform-specific gem on Windows and OS/X instead of searching for earlier Windows- or OS/X-specific versions.bundle install or gem install is run on a gem with Chef as a dependency. (PR)You can get it here.
You should see no changed behavior from 12.4.1, excepting potential bugfixes from dependencies which have been upgraded.
