We've Added New Tests: Kubernetes and GPC Tests

Since releasing our first test to assess cloud skills on Amazon Web Services, we knew we wanted to offer skills assessments for all the major Cloud providers. Lately, we've been working hard to do just that.

New DevOps Assessments Added to Our Testing Library

This month, we released two tests for Google Cloud: a short test for more junior engineers, and a more complex test tailored for mid to senior candidates.

We also decided to release an easier version of our most popular Kubernetes test that will provide quick validation of whether an engineer knows how to work with applications running on Kubernetes. This test was designed so it could be used by both software engineers and cloud engineers (DevOps, Cloud, SREs).

We added these junior-level assessments to our testing library because many of our clients expressed an interest in faster and easier skills assessments that would take only 15-20 min to complete, while still providing the same high-quality standard of assessing engineers' hands-on skills in certain technologies.

Read on to learn more about the new tests we are launching this month.

GCP Skills Assessments

Both Google Cloud Platform (GCP) tests provide candidates access to the Google Cloud console to work with cloud resources. Each candidate's test is created in a separate Google Cloud project, so their testing environment is completely isolated from others.

Introducing our new GCP Tests

Unlike AWS, Google Cloud doesn't allow the creation of usernames and passwords to access the GCP Console, so this means a candidate must have a Google account to log in to the GCP Console.

To overcome this limitation and to provide a seamless experience for candidates, we've implemented the Workforce Identity Federation. This allows candidates to use existing their Brokee credentials to log in to Google Cloud to complete skill tests.

This experience proved to be even more convenient than creating users on AWS, so in the future, we will attempt to achieve the same workflow for our AWS tests.

Similar to AWS and other tests on Brokee, hiring teams can still expect to receive the same detailed reports for each candidate. We will provide a history of user activity and AI summaries of the candidate's performance.

An example of a test report

Junior Kubernetes Assessment

Our most popular test for Kubernetes is often considered pretty difficult for mid-level - and even sometimes senior - applicants!

In addition to needing to know their way around Kubernetes (which is complex on its own), engineers need to understand how external-dns and cert-manager can be used together to automatically create DNS records and SSL certificates for the application running on Kubernetes.

We usually see one of three things when candidates take this test:

a) they solve the test completely in a reasonable time frame (1-2 hours),

b) they complete 70-80% but have a tough time

c) they can't do it at all.

Not all engineers who work with Kubernetes necessarily have (or need!) a deep understanding of Kubernetes administration. Some developers just need to be able to deploy simple applications, while their colleagues do Kubernetes administration.

This made us realize that a simpler test would be sufficient to filter out the majority of unqualified candidates, and checking core Kubernetes functionality would be more applicable to a wider pool of companies.

Introducing our Kubernetes Junior Test!...

We've launched our new K8s Junior Test

This test can be used for software developers or junior DevOps candidates (even though some would argue that there are no Juniors in DevOps 😈).

Candidates will fix configuration errors in a Kubernetes application to make it run correctly on the cluster. This should be a quick puzzle for those with at least some knowledge of Kubernetes.

---

Thank you for reading - we invite you to try out our new tests with your teammates or hiring candidates, and let us know what you think. What's coming up next month? We are developing skills assessments for Azure, stay tuned!

Read More: How to Choose the Best DevOps Engineer Assessment

Cloud Engineer vs Software Engineer: What Sets a Cloud System Engineer Apart

Cloud engineers and software engineers are often thought to have similar job responsibilities, but they actually have distinct roles in the world of technology.

While both work with software applications, cloud engineers specialize in the infrastructure and support of cloud computing systems, while software engineers focus on the development of software solutions.

Differences Between Cloud and Software Engineers

The main difference between cloud engineers and software engineers lies in their focus and skillset. Cloud engineers prioritize the management and support of cloud-based systems and infrastructure. They ensure the availability, reliability, and security of cloud services.

Software engineers, on the other hand, concentrate on software development, creating applications and solutions that run on various operating systems and meet specific user requirements.

Cloud engineers versus software engineers

What Is A Cloud Engineer?

A cloud engineer is a professional who specializes in designing, building, and maintaining cloud-based infrastructure and services. They have expertise in various cloud platforms and technologies, such as AWS, Azure, or Google Cloud.

Cloud engineers play a crucial role in modern IT environments, enabling businesses to leverage the power of the cloud for scalable and cost-effective solutions. They help organizations migrate their infrastructure to the cloud and ensure the smooth operation and ongoing support of cloud-based systems.

Cloud engineers are similar or sometimes, interchangeable, to the following roles: DevOps Engineers, Cloud Architects, Systems Administrators, Network Engineers, Site Reliability Engineers (SREs), Cloud Security Engineers, DevSecOps, Cloud Solutions Architects, Cloud Operations Engineers, and Data Engineers.

What Does A Cloud Engineer Do?

A cloud engineer is responsible for designing, implementing, and managing cloud-based solutions. They ensure the availability, performance, and security of cloud infrastructure and services.

Cloud engineers work closely with software developers, network engineers, and system administrators to create efficient and scalable cloud-based architectures. They are also responsible for monitoring and troubleshooting cloud systems, optimizing resource utilization, and implementing best practices for cloud infrastructure management.

Read More: The Top DevOps and Cloud Conferences to Attend in 2024

Are you on the cloud yet?

What is Cloud Computing?

At its core, cloud computing refers to the delivery of various services, such as storage, computing power, databases, networking, and software, over the internet, often through a network of remote servers.

This approach allows users to access and leverage these resources on a pay-as-you-go basis, eliminating the need for extensive on-premises hardware and infrastructure.

Cloud computing is characterized by its scalability, flexibility, and cost-efficiency, making it a driving force behind the digital transformation of businesses and the delivery of innovative services.

Cloud engineers play a pivotal role in the world of cloud computing by ensuring the seamless operation and optimization of cloud-based systems. They are instrumental in bringing the concept of cloud computing to life.

Cloud engineers come into play by designing, building, and maintaining the cloud infrastructure and services that organizations rely on, as well as sometimes even maximizing cloud financial management.

Job responsibilities vary between cloud engineers and software engineers

Cloud Engineers vs Software Engineers – Job Responsibilities

Cloud engineers are primarily responsible for designing, implementing, and managing cloud infrastructure and services to support an organization's IT operations. 

Their primary focus is on leveraging cloud computing platforms and technologies to optimize performance, scalability, and cost-efficiency. They ensure the reliability and efficiency of cloud services, working closely with cloud service providers such as AWS, Azure, or Google Cloud.

On the other hand, software engineers are involved in the entire software development lifecycle,  with responsibilities related to everything from designing and coding to testing and debugging. They create software solutions that meet specific user needs and enhance system performance.

Read More: Platform Engineering vs DevOps

Software Engineer vs Cloud Engineer: Salary Difference

In 2023 and beyond, both software engineers and cloud engineers find themselves in incredibly high demand in the technology industry. With the digital transformation of businesses and the growing reliance on cloud-based services, these professionals play a vital role in the development and maintenance of modern software systems.

The salaries for both roles are quite similar, and they depend on factors such as experience, certifications, technical skills, and proficiency in programming languages.

According to ZipRecruiter, the US national average salary for Cloud Engineers is approximately $130,000 mirroring the average for Software Engineers in 2023.

However, it's noteworthy that entry-level positions in the field of cloud engineering tend to offer a higher average salary at around $109,500, as compared to entry-level software engineering positions, which average around $91,966.

The broad pay range for entry-level positions in both fields, differing by as much as $48,500, implies that there are ample opportunities for career advancement and increased compensation, influenced by factors such as expertise, geographic location, and years of experience. 

Read More: The Best DevOps and Cloud Bootcamps of 2024

Salaries can vary greatly depending on location, skills, and more

Why are Cloud Engineers Paid So Much?

The demand for cloud engineers is high due to the increasing adoption of cloud computing by businesses. Cloud engineers possess specialized skills and knowledge in cloud technologies, making them valuable assets in the tech industry.

Their ability to design, build, and manage cloud systems that optimize productivity and efficiency contributes to their higher salary levels.

Keep in mind that expenses for hiring a cloud engineer go beyond just salary, but can include other hidden costs of your hiring pipeline, such as technical interviews, hiring mistakes, and more.

What Skills Does a Cloud Engineer Need?

A cloud engineer must possess a solid understanding of cloud technologies and platforms such as AWS, Azure, or Google Cloud. They should have expertise in networking, virtualization, and storage technologies.

 Additionally, strong programming skills in languages like Python, Java, or C# are necessary for scripting and automation purposes. Communication skills are also vital as cloud engineers often collaborate with teams and need to effectively convey complex technical concepts to non-technical stakeholders.

Is Coding Required for Cloud Engineers?

Yes, coding is an essential skill for cloud engineers. While they may not write large software applications like software engineers do, cloud engineers still require programming skills for scripting, automation, and infrastructure as code tasks. They often use programming languages to automate processes and manage cloud resources efficiently.

Consider coding languages like Python, Java, C#, Golang, and Powershell

Which Coding Languages Do Cloud Engineers Use?

Cloud engineers commonly use programming languages such as Python, Java, C#, Golang, and PowerShell. These languages offer flexibility, extensive libraries, and frameworks that enable cloud engineers to build and automate cloud infrastructure effectively. Additionally, proficiency in infrastructure as code tools like Terraform or Ansible can be beneficial in cloud engineering.

What Technical Skills Should Cloud Engineers Have?

Besides proficiency in cloud providers such as AWS, Azure, and Google Cloud, as well as being well-versed in programming languages to automate tasks and manage cloud resources effectively, there are other expertise cloud engineers should have.

A solid grasp of networking principles, including VPC, VPN, and load balancing, is crucial for building efficient and secure cloud architectures. Expertise in virtualization, storage solutions, and serverless computing is essential for resource optimization and scalability. 

 Additionally, knowledge of containerization technologies like Docker and orchestration tools such as Kubernetes has become increasingly important in a landscape where microservices and containerization are prevalent.

How to Evaluate DevOps or Cloud Engineers

When evaluating DevOps or cloud engineers for your team, it's important to go beyond traditional interview questions.

While scenario-based queries, technical problem-solving exercises, and inquiries about real-life experiences are valuable, modern tools like Cloud Assessments have proven to be more effective in assessing a candidate's performance on the job.

Read More: Choosing the Best DevOps Engineer Assessment

Brokee's DevOps assessments provide a real-world simulation of the tasks and challenges a DevOps engineer will face in their role. They offer a more comprehensive and practical way to gauge a candidate's proficiency in using tools, implementing best practices, and collaborating effectively within a DevOps or Cloud team.

By incorporating such assessments into your evaluation process, you can make more informed decisions about the suitability of candidates for the demands of cloud engineering roles.

Can a Software Engineer Work as a Cloud Engineer?

Yes, a software engineer can transition into a cloud engineering role with the right skills and training. Software engineers already possess the foundational knowledge of systems and programming, which is valuable in cloud engineering.

By acquiring additional skills in cloud technologies, networking, and virtualization, a software engineer can successfully transition to a cloud engineer role and take advantage of the growing demand for cloud expertise.

Choose the right type of engineer for your business needs

Which Engineer is the Right One to Hire?

Selecting the right engineer for your project or business is a crucial decision that hinges on the specific needs and objectives of your endeavor. When making this choice, it's essential to consider the nature of the project.

If you require infrastructure management, cloud deployment, and system optimization, a cloud engineer may be the ideal choice, bringing expertise in cloud platforms and technologies.

Read More: 7 Signs You Should Hire a DevOps Engineer

On the other hand, if your project centers around software development, application creation, and coding, a software engineer is likely the best fit, equipped with the skills needed for software design and implementation.

By understanding your project's unique requirements and the core competencies of each engineering role, you can make an informed decision to ensure the success and efficiency of your business or project.

We recommend using skill assessment software to qualify candidates, ensuring you choose the best talent for the job.

Read More: What's the Right Ratio of DevOps to Developers?

Final Word: Software Engineers and Cloud Engineers

Both roles are in high demand in the technology industry, with similar salary averages in 2023. The transition from software engineering to cloud engineering is feasible with the right skill set.

Ultimately, the synergy between software engineers and cloud engineers enables the creation of efficient and scalable cloud-based solutions, facilitating the digital transformation of businesses in the modern technological landscape.

---

FAQ

Q: What education and certification are needed to become a Cloud Engineer?

A: While a bachelor's or master's degree in computer science or a related field is beneficial, it is not always required to become a Cloud Engineer. However, obtaining relevant certifications, such as those from AWS, Azure, or Google Cloud, can greatly enhance your chances of getting a job in this field.

Q: What is the career path for a Cloud Engineer?

A: The career path for a Cloud Engineer typically starts with an entry-level position, such as an intern or junior Cloud Engineer. With experience and expertise, one can progress to roles like Cloud Architect, Cloud Consultant, or even Cloud Engineer Manager.

Q: What are the responsibilities of a Cloud Engineer?

A: The responsibilities of a Cloud Engineer include designing and implementing cloud architecture, managing cloud infrastructure, ensuring security and compliance, troubleshooting issues, and optimizing performance of cloud systems.

Q: How long does it take to become a Cloud Engineer?

A: The time it takes to become a Cloud Engineer can vary depending on various factors, including your prior knowledge and experience in IT or software development. However, on average, it can take several years of dedicated learning and hands-on experience to become a proficient Cloud Engineer.

Q: How much do Software Engineers and Cloud Engineers earn?

A: The salary of Software Engineers and Cloud Engineers can vary depending on factors such as location, years of experience, and industry. However, both roles generally offer competitive salaries, and with experience and expertise, the earning potential can increase significantly.

We Offer an AI-Powered Technical Recruiting Tool

Our team had a busy September. We expanded our team by hiring two engineers, an engineering intern, and a lead generation expert. Since it takes time to onboard new team members, we didn't plan too much for September. Despite that, we successfully delivered two features that we had been planning for a long time. Additionally, we moved our blog posts to a new platform called Ghost, which has improved the writing experience. Previously, we used Sanity, but I may write another blog post explaining why we made the switch. All in all, it was a productive month for us.

Long story short, let's jump to our big updates.

Recording Of User Activity

Brokee is a platform that offers technical skills assessments to potential hiring candidates. The candidates are given access to live IT infrastructure and are required to fix a broken application to showcase their technical skills with specific technical stacks. Previously, we provided a text-based history of the candidate's user activity that showed which commands they used to troubleshoot and fix various problems with the application's configuration, environment, or security.

Terminal History

This approach was helpful in providing a quick glimpse of how the candidate was attempting to fix IT infrastructure issues. It allowed us to see where candidates got stuck for 5-20 minutes and needed to spend time reading documentation or searching for potential fixes on the internet. However, it lacked some depth as we couldn't see what was happening when the candidate was editing a file. This made it difficult to judge if specific changes were relevant or if the candidate was just looking at a file without making any changes. We do provide automated evaluations of each test to ensure that the application is fixed at the end of the test, but it's still important to know exactly how the problem was solved.

So, we've integrated asciinema into our testing environment. Now, you can replay the whole user activity with a click of a button.

Asciinema Player

While this is great, nobody wants to spend 30-60 minutes rewatching the terminal activity of every hiring candidate, so we suggest the following approach for technical team members to spend the least amount of time analyzing user activity:

• Take a 1-minute look at the history of user commands.

• If you see something interesting or unusual, look at the time stamp, and jump to the specific time in the recording to understand what was happening in detail.

This way, you can quickly understand how exactly candidates solved specific problems in a fast and efficient manner.

AI-Generated Summary of Candidate Performance

Another major update on Brokee was the integration of AI into automated evaluations of candidate performance. While the history of user commands and replaying of user activity can help technical team members understand how every candidate fixes problems, every engineer has different skill sets and experience levels. As a result, less technical members of the hiring team may not be able to understand user activity, making textual and video representations insufficient to make informed decisions about the best hire. To address this, we asked ourselves how we could make the results more accessible to everyone on the hiring team?

Meet AI summaries of candidate performance:

As we are gathering data on user activity, we can use it to generate a structured summary with easily understandable explanations of what occurred in the testing environment. This allows everyone to quickly review the report and have a common understanding of which tools the candidate used to troubleshoot problems, along with their general purpose (such as networking configuration, working with files, security configuration, etc.). Additionally, the report highlights any changes made to the environment, and provides details on specific configurations used. As there are multiple ways to fix the same problem, even experienced engineers can learn something new by reviewing the AI-generated summary, especially if the candidate used a new tool to resolve an issue.

This is great because our automated evaluations can serve as performance reports as well as educational materials for the entire team. If you have multiple candidates who have completed skills assessments, you can refer to this report to identify the most efficient and creative problem solvers for issues with IT infrastructure.

Reporting Recap

This is just the beginning. We plan to provide more advanced reports in the future, but just to recap, here's what Brokee offers right now in terms of reporting:

• Test Completion Status: Passed/Failed

• Time Taken To Complete Assessment

• Number of Completed Subtasks

• Automated Checks

• AI-Generated Summary of User Activity

• History of Commands

• Recording of Terminal Activity

We strive to be your top choice in hiring DevOps and Cloud engineers. If you're planning to hire or need help assessing candidates, please contact us via our website, and don't forget to follow us on Linkedin and Facebook.

How Much Does Hiring DevOps Cost?: Hidden Costs of Your Hiring Pipeline

Hiring is a Top Concern for CEOs

Despite recession expectations, hiring tops the list of CEOs’ internal concerns. According to the 2023 Conference Board Survey, talent ranks first among internal worries for CEOs worldwide.

Yet only about one-third of U.S. companies report that they monitor whether their hiring practices lead to good employees; few of them do so carefully, and only a minority even track cost per hire and time to hire.

Tracking Hiring Costs

Over our time in the hiring sphere, we’ve spoken to dozens of CTOs, CEOs, recruiters, tech hiring teams, and HR leaders, and we’ve found that most companies don’t have data on the amount of time and resources they spend on hiring.

When we calculate hiring costs for them, most C-suites and directors are surprised and left wondering why they haven’t tracked hiring costs better to begin with, especially when they track other spending, from sales and marketing expenses, to travel and equipment.

By tracking hiring spending, businesses are able to allocate their resources better, save on unnecessary costs, and prioritize initiatives – such as better retention and candidate qualification.

Hiring Sys Admins, DevOps, and Cloud Engineers

This is especially pertinent when it comes to high-value DevOps and Cloud system engineer hires - some of the most expensive hires in the IT world.

Great companies spend immense resources searching for talented DevOps engineers, and even more to bring them onto the team. These recruiting and sourcing costs add up quickly.

But how does a company track the amount of time and resources they put toward Hiring DevOps Engineers? Let’s take a look at our Savings Calculator.

How Much Does Hiring a DevOps Engineer Cost?

We’ve created an algorithm that calculates DevOps and Cloud hiring costs, taking into account many factors:

•The Cost per Technical Interview, based on the salary of Senior Engineers who conduct these interviews.

•The Average Cost Per Additional Interview, which is based on the salary of HR or Hiring Managers who conduct cultural fit or other additional interviews.

•The amount of time it takes to prepare for, conduct, and evaluate each interview.

•The number of candidates that are interviewed to fill each role.

The costs for the interviews are calculated based on the average salaries in the United States for the respective roles. These salaries are then converted into hourly rates to determine the interview costs.

Read More: DevOps Assessments Streamlined EclecticIQ's Hiring Process [Case Study]

The Hidden Costs of Hiring

We will explain how this process works using our standard calculations based on the median US salaries from Levels. It's important to note that each company has the flexibility to adjust these numbers based on its own pay rates and specific hiring dynamics.

For our calculations, we consider the average hourly costs for a DevOps Engineer and a Software Engineering Manager, which are conservative rates.

Typically, a senior DevOps engineer spends at least onehour discussing candidates' technical backgrounds and asking AWS DevOps questions, while an additional hour is needed from the Software Engineering Manager, hiring manager, or team to assess culture fit and role expectations.

To hire a DevOps engineer, you would likely conduct a 1-hour technical interview with a senior member of the DevOps team. This amounts to 1 hour of their average salary, or $69 on average.

Keep in mind that this cost is per candidate, and typically, anywhere from 20 to 50 candidates are interviewed before selecting a hire.

DevOps teams often waste hours of time helping with hiring that could be automated

Let's say you had a pipeline of 100 candidates, 50 decided to take the test, and 20 passed the test.

In a typical hiring process, you'd have to interview all 100 candidates. By introducing an automated test, you save $6900 (100 candidates * $69).

Plus, your senior engineers will get an extra 100 productive work hours as they don't need to spend time interviewing those candidates.

Once the technical team qualifies a candidate, there is usually a second interview with a software engineering manager, costing approximately $139 for each hour of their time.

In our scenario, 80 candidates didn't pass to the next stage, so the hiring manager won't have to spend time conducting a second interview with those candidates.

This equates to an extra 80 productive work hours for your hiring manager and $11120 in savings (80 candidates * $139).

Based on the proposed rates, when hiring a single DevOps engineer, the hiring team would save an overall $18020 ($6900+$11120) as well as 180 hours of time.

This estimation does not include expenses such as HR and recruiting fees, onboarding, and other hiring costs.

How to Save Money on DevOps Hiring

By utilizing Brokee's technical evaluations, you can eliminate the initial technical interview for every candidate, resulting in time and cost savings for your DevOps hiring.

Furthermore, Brokee's tests have a 20% passing rate, guaranteeing that all engineers who pass the test will be highly qualified for the final interview with a software engineering manager.

By filtering out 80% of candidates who fail the technical screening, you significantly reduce the chances of low-quality engineers slipping through the cracks.

This saves valuable time for software engineering managers, CTOs, and other team members who are no longer required to interview these candidates.

Interested in saving money on DevOps hiring?

Brokee has multiple DevOps, Cloud, and System engineering evaluations for popular IT systems, such as Linux, Kubernetes, and AWS.

Our tests are ‘Chat-GPT-proof’ and ‘ungoogleable’, ensuring we’ll qualify only the best talent. Get started with a free demo of Brokee and start saving money while hiring with confidence.

Choosing the Best DevOps Test and Technical Screening Tool

From HackerRank Tests to CodeSignal and Codility Assessments

Selecting the most suitable DevOps assessment tool can be a daunting task, given the array of options available. From live interviews and paid assessment platforms to pair coding exercises, each method comes with its own unique format and set of advantages and drawbacks.

To help you make an informed choice, we've listed the most common DevOps tests and have provided insights from our DevOps expert, Maksym Lushpenko.

The Importance of Technical Assessments in DevOps Hiring

In the dynamic and ever-evolving world of engineering, the quality of a DevOps hire can make all the difference in the successful operation of your systems.

What is DevOps Testing?

Skills-based DevOps testing or assessments refer to the assessment and validation of an individual's skills and proficiency in various aspects of DevOps, including automation, continuous integration, and delivery, infrastructure as code, and collaboration, to ensure they possess the necessary competencies for DevOps roles.

DevOps and System engineers handle complex tasks, meaning the main goals of DevOps assessments are to efficiently filter out candidates who lack essential skills and ensure a fair and transparent hiring process.

Insights from a Senior DevOps Engineer

As a Senior DevOps Engineer with deep experience in hiring, I've had to give and take a DevOps technical interview or two (actually dozens!). Since I’ve been in charge of hiring DevOps and Cloud System engineers, I deeply understand the advantages and disadvantages of many types of assessments.

Assessments That Your Company Can Offer

Certifications help assess if a candidate meets the bare minimum

Screening Based on Certifications or Resumes

Screening based on certifications or resumes is a common method to evaluate candidates' qualifications and DevOps required skills. Screening allows employers to quickly identify potential candidates who meet minimum qualifications, possess relevant certifications or work experiences, and have some level of DevOps engineer skills.

Pros of Screening Based on Certifications or Resumes:

• Time-Saving

• Minimum Qualification Check

• Credential Validation

Cons of Screening Based on Certifications or Resumes:

• Limited Scope

• Lack of Context

• Credential Relevance

Take-home tests can add a burden for hiring teams to score

Take-Home Tests

Take-home tests are a DevOps quiz given to candidates as part of the hiring process. Candidates complete these tests in their own time and environment, addressing DevOps assessment questions and showcasing their DevOps engineer skills.

These tests allow candidates to showcase their skills without time pressure, providing employers with a deeper understanding of their abilities and problem-solving approach.

However, take-home tests can be frustrating and time-consuming for the hiring team, as creating and scoring tests can increase your overall DevOps hiring costs. Plus, it’s nearly impossible to ensure test authenticity since Chat-GPT or even another person can help a candidate cheat on their test.

Pros of Take-Home Tests:

• Flexibility and Convenience

• Realistic Evaluation

• Reduced Performance Pressure

Cons of Take-Home Tests:

• Potential for Plagiarism

• Time-Consuming Evaluation

• Limited Interactivity

Knowledge-Based DevOps Interviews

Knowledge tests or interviews for DevOps are your standard, real-time, question-and-answer format where the candidate reveals their knowledge about cloud platforms, technology, and coding, such as AWS DevOps interview questions. This allows the interviewer to assess whether they have the skills needed for DevOps roles.

Pros of Knowledge Tests or Interviews:

• Real-time Interaction

• Problem-solving Evaluation

• Personalized Assessment

Cons of Knowledge Tests or Interviews:

• Lacks a Real-World Element

• Adaptability

• Subjectivity

• Nervousness

Live coding or whiteboard interviews can help see if an engineer can think on-the-spot

Whiteboard Interviews or Live Coding

Whiteboarding or live coding is a dynamic evaluation method to assess candidates' problem-solving, coding, and algorithmic skills for DevOps roles. It helps evaluate whether a candidate can answer DevOps coding interview questions on-the-spot and if they have the skills required to become a DevOps engineer.

While live coding can be a quick way to see if an engineer generally knows what they’re doing, it can be insufficent to dig deep into an engineer’s day-to-day skills without a massive time commitment from both parties.

Pros of Whiteboard Interviews or Live Coding:

• Direct Evaluation

• Technical Clarity

Cons of Whiteboard Interviews or Live Coding:

• Performance Pressure

• Limited Context

• Subjectivity

• Resource Intensive

Assessments Other Companies Can Provide

Screenshot from a Brokee Live-Environment Test

Performance-Based or Practical Assessments

Performance-based testing is an evaluation approach that assesses candidates' practical skills and abilities in real-world scenarios.

Unlike traditional theoretical tests, performance-based assessments require candidates to demonstrate their expertise through hands-on tasks and projects directly related to the DevOps field - from understanding DevOps architecture to proving the use of various tools.

This approach is well-loved by hiring teams because it offers a more authentic and accurate evaluation of candidates' capabilities and suitability for the role.

Pros of Performance-Based or Live Testing:

• Authentic Assessment

• Skill Validation

• Engaging for Candidates

• Format Prevents Cheating

Cons of Performance-Based or Live Testing:

• Resource-Intensive

• Time Constraints

• Format is Less Open

Multiple choice tests work well for theory but are limited in testing hands-on skills

Multiple Choice Tests

Multiple-choice tests for DevOps are standardized assessments that evaluate candidates' knowledge of key concepts and principles in the field.

While these tests efficiently assess theoretical knowledge and cover various topics, they usually do not capture practical abilities or problem-solving skills and can allow for guessing or Googling answers.

Pros of Multiple Choice Tests:

• Efficient Evaluation

• Broad Coverage

• Objective Grading

Cons of Multiple Choice Tests:

• Limited Practical Assessment

• Guessing Potential

• Lack of Depth

• Potential for Plagiarism

Pair coding allows for live feedback sharing and provides an evaluation of teamwork skills

Pair Programming or Collaborative Coding

Pair programming or collaborative coding is an interactive approach to evaluate candidates' technical skills and collaboration abilities for DevOps roles. It involves candidates working with an interviewer or team member on coding or problem-solving tasks.

Advantages include comprehensive evaluation of technical skills and teamwork, real-world simulation of DevOps work, and immediate feedback. Challenges include potential subjective assessment, resource intensity, and candidate nervousness impacting performance.

Pros of Pair Programming:

• Comprehensive Evaluation

• Real-world Simulation

• Immediate Feedback

Cons of Pair Programming:

• Subjective Assessment

• Resource & Time Intensive

• Performance Pressure

Brokee offers more features for assessing DevOps than our competitors, at a better price

Q&A with a DevOps Expert

Now, for the fun part. We sat down with Maksym Lushpenko, Senior DevOps Engineer, former hiring team member, and the founder of Brokee, the first-of-its-kind testing platform with broken live environments for candidates to fix. Here are his thoughts about DevOps Assessments.

What is the Best Method of Assessing DevOps Engineer Skills?

As a long-term DevOps hiring veteran, I highly prefer and recommend the technique of Performance-Based and Live Testing.

Why Performance-Based, Live Testing?

I find that this is the best method because it offers a comprehensive evaluation of candidates' practical skills and problem-solving capabilities in real-world scenarios, making it the preferred choice for hiring managers who care about the quality of their engineers. Plus, Brokee streamlines hiring - both in terms of cost and time for companies.

Thoughts on In-House Assessments, like Pair Coding and Take-Home Tests?

I appreciate pair programming in some ways because you can share live feedback on a candidate’s coding style and see if they are open to feedback, however, it can be stressful for some candidates. For example, I’ve met some engineers who are extremely talented when working individually but struggle in a pair programming environment.

Take-home tests, in my experience, are usually frustrating and time-consuming for the hiring team, as creating and scoring tests can take many hours of valuable work time. Plus, it’s nearly impossible to ensure test authenticity since Chat-GPT or even another person can help a candidate cheat on their test.

What Makes Brokee Different from the Competition?

As the CEO and Founder of Brokee, I’ve created a Performance-Based Live Testing Platform to take this method of assessment to the next level. Brokee stands apart from competitors, like HackerRank assessments and DevSkiller tests, by offering automated, affordable, and easy-to-use live broken environments that are Chat-GPT and Google-Proof.

Why Choose Brokee?

• Authentic and Accurate Assessment

• Interactive and Engaging

• Efficient and Uncheatable

• Helps companies save time and money

What are Other People Saying About Brokee?

Brokee is way better than using coding challenges or multiple choice questions for infrastructure oriented roles - Milton Torasso, DevOps Engineer at Kiwibot.

We are a performance-based remote hiring platform, and by seeing how candidates perform on Brokee's hands-on real-work, real-environment tests, we can predict their on-the-job performance. Brokee's support and communication have been amazing since day one. - Co-Founder of Flatworld.

Brokee's test evaluates your hands-on experience with issues that could occur in a real production application in k8s. Luis Nuñez, Senior Software Engineer at GitHub.

The test is interesting; while taking it, I discovered new problems that I wasn't yet familiar with. Brokee's tests are not just a knowledge check but serve as independent training. Yulia Krasikova Senior DevOps Engineer at Accenture.

Final Word

The success of the hiring process hinges on understanding the unique goals and requirements of each organization.

Whether it's Maksym, our DevOps pro's favorite - Performance-Based testing - or other methods like pair coding or knowledge tests, choosing the right assessment method should be based on the specific demands of the position and company culture.

By choosing the right DevOps assessment technique, you can ensure that the best talent possible joins your ranks, which will drive innovation, efficiency, and success in your company.

We’re Offering Free Service to Ukrainian Companies!

Photo by Max Kukurudziak / Unsplash

Our Offer to Ukrainian Companies

We will provide free Brokee testing services on a first-come, first-served basis. The number of places is limited and will depend on the available funds and demand.

Our Mission:

As a leading tech hub in Europe, Ukraine boasts an impressive tech workforce of approximately 300,000 professionals. This thriving tech ecosystem is home to global companies like Grammarly, GitLab, and Wix, which were all founded in Ukraine.

The highly skilled tech professionals in Ukraine are making a significant impact on the global economy and technology landscape through the innovative solutions they create.

We at Brokee are proud to be a Ukrainian-built company, and much of what we do is inspired by our support for the Ukrainian tech sector. Recently, Brokee launched an initiative to offer our solution - which is an alternative to DevOps engineer assessments - for Ukrainian companies free of charge.

Who Qualifies for Free Brokee Testing Services?

- Ukrainian startups, especially those still in the very early stages of development and who have not received funding.

- Ukrainian IT companies or IT recruiting agencies affected by the war.

- IT schools with a social component - if you are teaching people for free or helping Ukraine in some other way, we are happy to help you.

- Large Ukrainian IT companies: Brokee’s services are affordable, so we do not expect that payment for our services will be a problem for large IT companies. However, we are happy to provide a free trial, especially for companies that donate to Ukraine’s defenders or other Ukrainian charities.

What is the Duration of This Offer?

We plan to provide free testing for qualified companies for approximately six months, or until Ukraine’s victory. On average, we help reduce DevOps hiring costs by over $3000 and several weeks per role.

Brokee is a startup currently without funding, and I personally have been using any profits and my own savings to keep it operating.

I believe in Brokee and what it can provide companies, so I developed this program to support Ukraine's tech sector. Our service has helped multiple companies streamline their DevOps hiring.

How to Get Started

Book a call with us using this link or fill out the contact form on the Brokee website. If you are a DevOps or Cloud Systems Engineer and would like to try out challenges - such as Amazon DevOps, GPT, Linux, Ingnx, and more - please indicate which test you are interested in and we will send you an invitation.

For anyone interested in collaborating or working with Brokee, don’t hesitate to reach out.

Slava Ukraini!

Photo by Yurii Khomitskyi / Unsplash

Case Study: DevOps Assessments Streamlined EclecticIQ's Hiring Process

About the Client

EclecticIQ is a global provider of threat intelligence, hunting, and response technology and services, headquartered in Amsterdam. Founded in 2014, EclecticIQ is a top European cybersecurity vendor operating worldwide with offices and teams across Europe, North America, India, and via value-add partners.

Client History

Brokee’s CEO, Maksym Lushpenko, had worked previously at EclecticIQ as a Senior Cloud Engineer. During his tenure at EclecticIQ, the cloud operations team faced a major challenge: evaluating the skills of DevOps and Cloud system engineers without wasting the time of their Engineering Team.

In 2021, EclecticIQ was the first company to adopt Brokee’s innovative DevOps assessments to evaluate their system engineer candidates. Since then, Brokee’s platform played a key role in the way EclecticIQ hires cloud engineers and technical support engineers, optimizing their hiring process.

The Problem

Just like many other IT companies, EclecticIQ's DevOps teams were overloaded with work. They faced a conundrum: The team needed to hire more engineers to share the workload, but qualifying and hiring new engineers took a huge chunk of time from the team members' already limited time.

Read More: How Much Does Hiring DevOps Cost?

For EclecticIQ, hiring one DevOps engineer takes more than 45 hours of a senior DevOps engineer's valuable time. This includes the time spent on individual DevOps testing for every candidate and subsequent group discussions with those who pass the initial technical screening call. This could cost a Dutch company – like EclecticIQ – up to $2500¹ in employee hours to fill one role.

Plus, due to the critical role DevOps engineers play, scheduling interviews around their schedules can be a formidable task for companies, given the demanding nature of their work.

The Solution: DevOps Assessment Software

Brokee provided EclecticIQ with an efficient way to screen DevOps candidates, making their hiring process fast and simple.

DevOps Assessments for Better and Faster Hiring

Brokee provides a platform for evaluating candidates on needed DevOps engineer skills. Compared to other DevOps Engineer Assessments, each of Brokee's test truly reflects the day-to-day tasks that engineers regularly work on, by using broken environments that they have to fix. Passing one of Brokee's evaluations proves that the candidate has the needed DevOps skills for the job.

Brokee's DevOps testing platform was an ideal solution because it freed up the DevOps team's time, allowing them to work on critical projects, while helping to hire engineers who could jump right in to help ease the team's workload.

EclecticIQ DevOps Team employed Brokee's Kubernetes with Nginx ingress Assessment to evaluate potential DevOps candidates, expediting the process of hiring essential team members.

The Word is Spreading: Skill Tests Result in Better Hires

EclecticIQ's Technical Support Team heard great things about Brokee from their DevOps team, so they also expressed interest in adopting this efficient approach for their hiring process.

We developed a Linux Assessment based on the DevOps assessment questions that the Technical Support Team normally asks during technical interviews. The Linux Assessment has already yielded success by helping the Technical Support Team qualify and hire several talented support engineers.

The Benefits

Through Brokee, EclecticIQ has already hired several high-performing Cloud and Technical Support Engineers.

These engineers managed to complete Brokee’s tests and immediately showed themselves to be proactive and skilled workers on their teams, which reduced the amount of time the new hires and the teams spent on technical onboarding.

New hires were able to take on projects independently and help their colleagues with difficult tasks from day one, thanks to Brokee's highly applicable live environment.

When a candidate completes Brokee's test – which takes about one hour – the results are sent to HR and the relevant hiring team, saving all parties the whole first week of interactions.

EclecticIQ gained multiple benefits from using Brokee

By using Brokee’s DevOps Test, EclecticIQ benefitted from:

• Standardized, consistent, and bias-free approach to technical assessments

• Comprehensive candidate reports are automatically sent to Slack and Email

• Hiring risk reduced by 47%

• 2x reduction in tech interviews

• Saved HR and Tech Hiring Teams the first week of interactions with candidates

• $2500 worth of Senior DevOps hours saved per role (45 hours)

• Candidates who pass the test should be able to adequately perform DevOps work from day one

• Shorter technical onboarding time

• A simple, clean, and easy-to-use evaluation platform

• Less frustrated DevOps teams

• Tests for Kubernetes, Amazon DevOps, Linux, GPC, Inginx, and more

Conclusion: DevOps Assessments Boost EclecticIQ's Hiring Process

We have greatly enjoyed growing side-by-side with EclecticIQ, and are excited to continue working with them in the future.

Brenda Szongoth, the Talent Management Partner at EclecticIQ, said she was very satisfied using Brokee due to its "usability, great customer support, and relevant selection tool".

Brokee's platform has good usability, great customer support, and relevant selection tool.

We've been told by the teams at EclecticIQ that the performance and quality of the candidates EclecticIQ hired through Brokee have been “excellent”.

If you are interested in a faster and better way to hire DevOps, Cloud, or System engineers, Brokee will help you improve your process. Please contact us to get started saving time, money, and peace of mind when hiring.