Tech stacks for DevOps Engineer

Currently I am thinking what kinds of technologies stacks a professional DevOps engineer should own? In other words, what kind of tech statcks can empower DevOps engineer to handle their infrastructures/development issue?

Let’s see a famous picture, http://www.oracle.com/technetwork/server-storage/solaris11/overview/whats-new/engineered-for-oracle-518998.html 

engineered-for-oracle-1352069Basically, a DevOps engineer should be able to handle,

1. Application issue – administration, applications build/release, etc.

2. Middleware – administration, deployment, etc.

3. Database – administration, internal tools, etc.

4. OS – MSWins, Linux, Unix, etc.

5. Virtualization – Templatizes.

Currently my technologies stacks:

1.     Oracle OCA/OCP certificates – Database

2.     WebLogic Administration Certificate – Middleware

3.     SCJP  certified – Sun Certified Java Programmer –  Language

4.     SCWCD certified – Sun Certified Web Component Developer – Web

5.     EBS AppsDBA Certified –  Apps Administration

6.     LabManager, VirtualBox experiences

This stacks ensure that I can handle my daily routine/urgent issues very timely and precisely. I think I can enhance myself in AWS, cloud services, etc.

 

Process of killing an exam in general:

1. Get oral approval from your immediate manager. (You need to convince s/he that your exam will benefit your daily work).

2. Prepare the exam.

3. Try to apply voucher from http://education.oracle.com/pls/web_prod-plq-dad/db_pages.getpage?page_id=156

4. Contact local exam agent to take exam. If you have a voucher, you can have 30% discount.

5. Pass the exam.

6. Get reimbursed with your exam transcript and exam receipt.

 

How to prepare: 

OCA (it is prerequisite of OCP) :

OCP (it is prerequisite of AppsDBA):

AppsDBA:

WebLogic:

SCJP (it is prerequisite of SCWCD):

SCWCD:

 

For OCA/OCP:

OCA/OCP Oracle Database 11g All-in-One Exam Guide with CD-ROM: Exams 1Z0-051, 1Z0-052, 1Z0-053 (Oracle Press)

For WebLogic:

Oracle WebLogic Server 11g Administration Handbook (Oracle Press) (Osborne Oracle Press Series)

For AppsDBA:

(Going thru EBS 12.1 Administration doc is good enough).

For SCJP:

SCJP: Sun Certified Programmer for Java Platform Study Guide: SE6 (Exam CX-310-065)

For SCWCD:

SCWCD Exam Study Kit: Java Web Component Developer Certification

 

How To Succeed At Working From Home

In the past two years, I worked from home sometimes. To be honestly, I do prefer working from office however I work for a very distributed release team and I am the only member from China.

For me, Release Engineering is a 24*6 job and working from home can give me more time to work especially in a developing country the traffic jam is very terrible. The lucky thing is, in the past two years, I could contribute more to the team by working from home and I want to share my experience in working from home from perspective of Release Engineering.

Actually this post is based on http://oduinn.com/images/2012/blog_we_are_all_remoties.pdf with my real experience. In John’s blog, http://oduinn.com/blog/category/remoties/ it has some more very good posts about how to be a good remotie in Release Engineering.

(I am on a long vacation from Sep 14th to Oct 7th. Should continue this topic after I am back from vacation 🙂 )

 

############################################################

I happened to get a survey about how to be a good remotie. Here is my input,

1. What are the biggest challenges in your distributed team?
2. How have you tried to overcome these challenges?

a.     Working hours/Time zone.

My team members from India are working in IDC+5.5 timezone which are 2.5 hrs later than China time while other team members from San Jose work in Pacific timezone. For example, my team members from India usually come to office around 10AM (12:30PM China time) and others from San Jose start to work around 9AM Pacific time (23:30 China time). Particularly we have different Public Holidays Schedules and also different vacation plan (people usually take long vacation by conjunctting with their public holidays so in that way they can enjoy a long vacation with less leave days).

My way to overcome this:

If that day I work from home, I usually get up at around 7AM China time to check my email in case any requests/issues from US (It is around 5PM Pacific Time). If no issues, then I will hit my bed again to take sleep otherwise I will start to work. Usually I will take 1-2hrs to hae breakfast, walk around the park, take a bath, or go to library (my apartment is near the district library) before 11:30AM, and come back to work at around 12:30PM, when team members from India are arrival to office. My main working hours will be 12:30PM – 8:00PM (7.5hrs, it is 10AM – 5:30PM India time) with 1hr leave for lunch at around 2PM China time. I will come back to work at around 11:30PM China time to work with US team members when they are arrival office at 9:00AM their time). In this way I can work around 8-9 hours only but I can cover all of the issues from a very distributed team in a very timely manner. Regarding holidays and vacations, we usually create an info page to note down all of the public holidays in different offices for our references, so that we can do backup/handover before our holiday and we can leave for even 1month without any issues.

My suggestion here:

If someone wants to be successful in a very distributed team he has to be a very autonomic guy and generous. Without managers’ monitoring, it is not easy to work professional as you have too many chances to kill time. Don’t count working hours too much because working in a distributed team you have to get yourself prepared for issues/bugs/requests anytime. But also it is a very good welfare that you can work remotely because you can work even from your living room and you can take sleep sometimes if no urgent issues.

b.    Team spirit.

Due to team budget, we can’t travel to other office to meet each other face to face. Sometimes I might feel very down suddenly when I realize that I am working alone from China. This feeling is very terrible and I might have idea to have a job change.

My way to overcome this:

During time slot, I chat with my team members for a while about entertainment, learning local languages, etc. In this way I can sync up with them just like we were in the same office and were talking in office canteen. Last year I also had personal vacation to India and enjoyed my vacation there.

My suggestion here:

The company should give some budget for team travelling at least once every year. It is not for any technical issues but just to ensure team members from a distributed tam can see each other annually. For example, this year it can choose City A as the destination while next year it can change to City B.

 

                 Overall, I am very comfortable to work with such a distributed team.
3. What have been the key factors to ensure successful collaboration in your distributed team? (processes, competences, values, behaviour, habits, tools etc)
    1. Define the processes. Our build/install automated flow has to be run overnight for couple of days, we have to incorporate the helps from other offices so every day before we go offline we will send out status email and hand over the work to other team members in other time zone. In the email we will brief our status and todo list.
    2. Every member in a distributed team should have a cell phone from that they can check emails regularly and also could be able to connect to IM from phone. This is very critical to ensure that team members can response very quickly – this is not a kind of sacrifice but it is a kind of compensation from team members to the company as the company also gives its team members enough trust and time to get a better work & life balance.
    3. Responsible, good in communication, and high self autonomous. Working in a distributed team means you have to do work hand over via email or phone but not communicate face to face. You have to be good at communication, at least, have skills to make your email clear and hit the points. Be responsible, the working day is not end after you send out the hand over email. We are not in the same office, if you are disappeared just after sending out the mail then supposed that people from other office hits issues and needs your input in urgent?
4. Please describe leadership in your team (who is perceived as the leader(s), how actively is he/she involved etc)
5. Please describe an ideal leader of a distributed team? (competences, values, behaviour, habits etc)
    1. My manager trusts me. We enjoy high trust between each other. He never question me where am I and what am I doing? He gives me the most independence as he can and does his best to make me feel comfortable. For example if I have to on leave for half a day in urgent, I just need to give him a call or ping me via IM. And I am also no need to submit leave application via our HR system because my manager will reject that as he says ‘I know you work hard during weekend and nights sometimes, so pls. just regard this leave as compensation’. In a distributed team, neither of manager or employee himself should count working hours too much as in such a team flexible working hours should be adopted. As a manager, should not question his employees’ questions like, where are you? What are you doing? Or ‘I am not able to contact you’, etc.
    2. My manager gives me enough challenge. By providing employees enough challenge, managers can keep their team members focus on their work otherwise they may get used to their work and get bored without inspiration. Take me as example, my manager always knows when he should give me new challenge and this kinds of new challenges can make me feel fresh in the team. This is also a way to feel the team spirit.

Release Engineering as a Force Multiplier

In this post, I will post my understanding and its keynotes about http://oduinn.com/blog/2013/06/26/release-engineering-as-a-force-multiplier-at-google-techtalks/ , http://oduinn.com/blog/2013/06/07/release-engineering-as-a-force-multiplier-relengcon-2013/

Nowadays, Release Engineering is important to the success of every software project. People might find that more and more companies provide positions in Release Engineering and I am optimistic that there will be more and more people become professional Release Engineers in future. I am surprised that the world’s first ever Release Engineering conference was held in SF on this May 20th 2013! The industry should have realized the value of Release Engineering earlier!

Building an effetive pipelines (in my current company, we call it ‘flow’) requires a different mindset to writing a great software product. Release Engineers != Developers. I ever interviewed with some candidates which have good background in coding however they seems don’t have a good sense in Release Engineering. In their mindset, they just think that release engineering is just a job writing shell/perl scripts and nothing much! Hmmm, I understood that it is very common as Release Engineering is not taught as a discipline in any computer science. When I graduated from university in 2009 and started my career as a Release Engineer, I ever had any sense of Release Engineering. In my first 2-3 years, I had to learn on-the-job. This limits the effectiveness of Release Engineering to what can be learned on the job, and because information isn’t widely shared, its hard to learn from other people’s mistakes or successes. I am planing to write a booklet about Release Engineering to introduce the concept of Release Engineering, how it works, and what tools/scripts they will utilize, and how to be an seasoned release engineer, etc. That is also the reason why I opened this blog – I want to introduce Release Engineering to others. 🙂

When an organization encounter that its software delivery is slower and the product qualities are worse than before, and find that increasing headcount in dev & QA could not ease this pain. Yeah, show off time of Release Team. Release Engineers make entire organizations more effective while dev make good quality software. In the keynote, the speaker lists below data from his org (Mozilla):

Before & After

How quickly can we ship a chemspill release?

4-6 weeks 11 hours

How long to ship a “new feature” release?

12-18 months 6 weeks

How many active code lines?

1 1/2 42

How many checkins per day?

~15 per day 325 per day

I also have similar experience in my projects. We have nightly builds, checkin builds, branch builds, mainlines builds for different platforms and versions. Without this kind of build & release support, our dev & qa could not focus on their daily activities. Besides build & release, I also provide internal automated tools for them.

Who are Release Engineers’ customers? Haha, interesting topic. In the keynote, speaker provides two kinds of customers, Founder(s), CxO and VP Eng. Think those people are too big to us? Let me share you a real lesson. In my first job, I had ever done an info sharing session with one of our big boss who came from Headquarters by sharing her about Release Engineering team and how it works. However I had only 1.5 years working experience and I was still a newbie in Release Engineering. I had a bad presentation in this session because in this session, I shared her about what tools/ what techonoliges we used however seems she was no interested in that. In next paragraph, I will show what they are interested in. Here, I just want to convey that as a release engineer, you are closer to upper management than you think. Upper management they don’t care what tools/languages/technologies you are using, what they concerns are, how fast software can be delivered and how are the product quality. Release Team is in a good position to answer these kinds of concerns. Besides upper managements, we also have to walk around, chat with Dev+QA and find what do they care about and what do they need? And then do our best to comfortable them and improve their productivities. Basically we should be able to answer below problems (They are also the things what does upper management care about):

• fallout from shipping bad release
• fallout from slow response to security problem
• losing customers because of buggy software

missing release schedules during feature wars
(webM, …)
• turning away potential revenue
• infrastructure limits to hiring more developers

http://oduinn.com/blog/ is a very good source to learn how Release Engineering runs in a source organization.

 

IT Certifications in RE

In this post I will share my experience in IT Certifications and what certifications are good for Release Engineering, and also I will introduce my experience in preparing the exams.

Let me explain why we need to take IT Certifications in our careers.

As a release engineer, we have to involve into a lot of technologies and it is impossible that we can master all of them. We release engineers are generalist in technologies stacks but not a specialist in one or two technologies only. However we can still pick up some of the important technologies to study. For example, if in your daily work you have to do a lot of works pertaining to Oracle Dataabse, then why not prepare exams to take a certification in Oracle Database? It is a very good way that study by taking exams and it is a also a good way to have a systematic study.

Every big IT company will have their certification exams and paths. Below I will take Oracle Company as example,

For Oracle University, it has 8 categories exames.

1. Industries

Oracle Retails, Oracle Insurance, Oracle Tax Applications, etc. I think exams in Industries are more functional-related so I don’t recommend you to take.

2. Applications

Oralce EBS, Oracle Fusion, Oracle Siebel, etc. Here it is not too much functional related but an Apps DBA role. I recommend you to take one if you have time. For myself, as I am working for EBS, then I got myself Oracle EBS AppsDBA certified. This helped my daily work in EBS system administration.

3. Java and Middleware

Java Programmer – SCJP, Java Web Component Designer – SCWCD, etc. Here it is more development and I strongly recommend you to take one or two exames here. For myself I took Java Programmer and Java Web Component Designer certified and they helped me a lot in my daily build/release activities. I don’t think now I am a professional Java developer but I am a very professional build engineer in any OO-languages applications! Hereby, I am reiterating here that a certification in this category will definitely help you do a good job in Release Engineering. I also have plan to take a certification in Weblogic.

4. Database

Database Assistant, Database Professional, SQL developer, etc. Nowadays all of the applications are based on a rational database so I am strongly recommending that you take one certification in Database. For myself, I took the Oracle Database Professional Administrator.

5. Operating Systems

Oracle Linux Administrator, Solaris Networking Administrator, etc. I am preparing a Linux Administrator exam.

6. Virtualization

Oracle VM. I think it is IT job and not necessary to take.

7. System

Exadata, Servers, Storage. I think it is IT job and not necessary to take.

8. Foundation

Oracle IT Architecture.  I think it is IT job and not necessary to take.

How to prepare a certification?

1. Firstly visit the exam website to know the exam contents and the certification path. For example, for EBS AppsDBA certified, it requires a certified in Oracle DBA firstly. So if you want to get EBS AppsDBA certified, you need to prepare the Oracle DBA firstly.

2. Prepare material. You can buy a book from online bookstore. Basically you need to spend 2 months to understand the material.

3. Search for an exam dump from internet after you finish the material and just two weeks before you take the exam. It will help you pass the exam.

4. Take the exam.

Material recommended:

1. for SCJP:

SCJP Sun Certified Programmer for Java 6 Exam 310-065

It is very easy to find an exam dump for SCJP. But for myself I didn’t need a dump to pass the SCJP exam as I had enough time to prepare it because I passed it when I was at university.

2. for SCWCD:

Sun Certified Web Component Developer Study Guide (Exams 310-081 & 310-082) (Oracle Press)

I didn’t see a dump in internet for this exam. But for myself I also didn’t need a dump to pass the it as I had enough time to prepare it because I passed it when I was at university.

3. for OCA/OCP:

OCA/OCP Oracle Database 11g All-in-One Exam Guide with CD-ROM: Exams 1Z0-051, 1Z0-052, 1Z0-053 (Oracle Press)

It is very easy to find a dump about OCA but I didn’t see a dump for OCP in internet. I passed OCP with scores 70% and the passing score is 66%. Almost died.

4. for EBS AppsDBA:

I didn’t have a book or dump for it but it is very easy and all are related to my daily work so I passed it with a fly colour score. 🙂

Go, for certifications. 🙂

How to win a good appraisal as a Release Engineer

In most of the companies, they will have 5 ratings in their appraisal system.

It is 1, 2, 3-Successfully meets expectations, 4-Exceeds expectations, 5-Outstanding.

And it will have the same competencies for every employee. like Business Ethics, Innovation, Teamwork, and Project Management.

for the objectives, they are different from everyone.

I don’t want to discuss what’re those competencies meaning as they are very frank. Here I will just share my first three years after my graduation from campus.

First year:

I got a rating with 3-Successfully meets expectations. Actually this was also the same as my expectation since in the first year I didn’t have too many chances to perform. I think this is very common for every newbie in career. So my suggestion here is, in your first year, don’t expect too much but just work hard and learn with a good attitude. As a release engineer, you should learn how to build a project from inception. How to setup a build environment/machine. Get a basic sense about continuous integration and delivery. Knowledge in SCM tools. And fall in love with Release Engineering.

Second year:

In my second year in career. I got a rating with 4-Excellent. To be honest, I did think I should have got a 5-Outstanding. But my manager told me that my hard working and diligent did impress her but she said that it was not enough to get a 5-Outstanding as my work was only helping the Release Team only. She said I should extend my responsibility outside of Release Team. So here I learnt a lesson is that, as a release engineer, you can not just focus on your release activities, you should seek chances to involve more into other aspects like testing, development, architecture, even requirement management. Here my suggestion is, in the second year, a release engineer should have nice scripting ability and should have automate as much as possible. And should have deep understanding of CI, take part in project management from inception to deployment. Also should enhance your tools experience, like JUnit, DbUnit, Findbugs, PMD, etc.

Third year:

In my third year in career I finally got a rating with 5-Outstanding. In this year, I not only could complete  my Build/Release tasks with a high satisfaction but also could help my team members in Relase Team as a problems killer, besides that, I could also help others from other teams like Testing, Platforms. In this year, I enhanced my skills like Database Management, Linux, etc. And I could work with a very effective way and always eager more. Worked with good attitude and detailed in analysis.

Above is my career path in the first 3 years. I hope it could help students who want to join in the Release Engineering. In a word, no pain no gain. RE is a very interesting field and it deserves your dedications.

Starting from your third year:

You should be over qualified for any release activities for any projects. Besides that, you should be passionate in how to develop an integrated platform to automate and centralize your automated scripts/reports, and upgrade them to be the core utilities for developers.

Here is a conclusion:

first year: from a newbie to a junior release engineer. Knowledge in SCM & CI. Tools slave. Learn scripting. Can do attitude.

second year: Good knowledge in SCM & CI. Tools expert. Can help other members in Release Team.

third year: Problem killer. Can help others outside Release Team. Be a generalist.

Fourth year and going on: working on integration infrastructure.

Welcome to Release Engineering! 😉

Career Path – Release Engineering

Release Engineering (“RE”) is a sub-discipline in Software Engineering concerned with the compilation, assembly, and delivery of source code into finished products of other software components. It is Release Engineering that making Software Engineering as manufacturing is to an industrial process. We can say that without RE, the Software Industry was still in primitive society.

Let me take Oracle EBS as an example,

How many product family (every product family will consist with around 10 small product) Oracle EBS have?

Oracle CRM
Oracle Financials
Oracle HRMS
Oracle Mobile Supply Chain Applications
Oracle Order Management
Oracle Procurement
Oracle Project Portfolio Management
Oracle Quotes
Oracle Transportation Management
Oracle Warehouse Management Systems
Oracle Inventory.
Oracle Enterprise Asset Management

And in the techology stacks, it has to support Linux-32/64, Solaris-32/64, AIX, HPUX-IA64, Windows; Weblogic, Oracle iAS, Oracle Forms, Oracle OC4J, Jrockit, SUN JDK, and comprehensive support plans for different customers, different release and different upgrade paths.

You can have a good understanding how complicated in EBS from,

http://www.oracle.com/technetwork/apps-tech/2011-june-ebs-roadmap-405789.pdf

https://blogs.oracle.com/stevenChan/

As software development evolves, it is more difficult and even impossible that without a professional release engineer in a software development team and they can succeed to provide good quality products in time. Actually I am a passionate practitioner of Book <Continuous Integration: Improving Software Quality and Reducing Risk> and book <Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation>. I strongly recommend you to read these two books if you are interested in RE.

An Release Engineer’s daily job,

1. Code Management,

It has lots of good code management tools, like CVS, SVN, Perforce, even with Linux’s RCS. Here has to work as an SCM.

2. Mainline/Branch management,

Manage mainlines, branches, branching strategies and the bugs attached to them. Here was like a product manager 🙂

3. Install/Upgrade Testings,

Install testing, fixes, bugs, and verify all of the possible upgrade paths for customers. Here works as a system testing engineer.

4. Report/analysis/triage any build compile issues,

Here works as a debuger. Find out who do the evil and expedite fixes. There are some nice build tools like Maven, Nant, ant.

5. Code qualities,

Leverage JUnit, Findbugs, PMD, Checkstyle, Sonar to enhance code quality. And it is easy to plugin them into a workflow build engine like Hudson, Cruise. Here works as a Quality Controller.

6. Integration issues,

How to integrate a product suite with a complicated tech stacks is a big challenge nowadays.

7. Virtualization,

Utilize VM tools to templatize the latest artifacts into templates. Works as IT guys.

8. Internal tools enhancement,

Release Engineers have to get ready to product tools to enhance productivity. Works as developers.

9. Database management/OS management

We can see that as Release Engineer, ppl have to involve into OS, Database, Middle tier, Languages, Build/Release, Testing, Code quality, IT, even requirement management!

So in my opinion, how to become a professional Release Engineer?

1. Firstly, you need to love the Software Industry and take it as your interesting but not merely a job!

2. Release Engineer is a Dev + Ops. (DevOps) + Test + IT. You need to develop your skills as a generalist.

3. Enhance some key skills, like Database, Mid-tier services, development skills. (That is why I took Oracle Database, Java certified).

4. Detailed in analysis, and always ready to help ppl. You have to have development sense and get ready to work after office hours sometimes as the build will fail in the midnight and get it fixed before testers yell to you in next morning.

5. Think from customers’ view sometimes. You are the last chance to find out issues before rolling out the products.

6. Be responsibility. When a build issue, exception, bugs occur, do NOT throw it to Dev/Tester right away although you can do like that. You should have a detailed and quick analysis that firstly.

Career path:

Release Engineer -> Have sense in Release Engineering. Good knowledge in tools/Scripts/tech stacks.

Sr Release Engineer -> Can do detailed analysis and deep knowledge and skills. Conduct Release plan.

Release Lead/Manager -> Compose Release plan and lead the team.