The internet is all abuzz about ChatGPT. Have you tried it? This AI application, part of the generative artificial intelligence category of tools that generate text, images and sounds based on a user's prompts in a conversational manner, is finding new uses every day. More than a million people are using it to create everything from poetry to high school term papers to rewrites of Queen songs. In fact, that usage has stretched the boundaries between original content and AI generated material. Honest, ChatGPT was not used to create this post.
Usage has also stretched the value of OpenAI, the developer of ChatGPT - Microsoft is reportedly in negotiations for a $10B investment in the company. Additionally, there are rumors that Microsoft's Bing will incorporate ChatGPT to power its search results as soon as February. That is blazingly fast, given that OpenAI launched the current version of ChatGPT at the end of 2022.
ChatGPT has become a timesaver in areas such as search, writing, coding and design. Although the quality of its writing and coding may be considered mediocre and generic, many people find it good enough and perhaps, even better than their own output. ChatGPT has even been used to write a script for a commercial with actor Ryan Reynolds.
Coding with ChatGPT?
So if people are willing to pass off ChatGPT content as their own, what does that mean for those of us whose job it is to evaluate content, whether articles and reports or even computer coding? Especially since the articles seem to credible and facts verifiable. In fact, it is quite difficult to identify content as AI created. Coding with AI and ChatGPT has started to become viable, so much so that Gizmodo believes that Computer Programmers could be at risk to lost their jobs to AI.
Experts have suggested that "College as we know it will cease to exist." In fact, earlier this week the NY City public schools banned the use of ChatGPT from school devices and wifi. The concern is that the tool will stunt learning with students cheating on essays and tests.
In recruiting, many have started to use ChatGPT in their daily work by creating job descriptions, sourcing strategies, email outreach and even interview questions. Of course, if you are going to rely on ChatGPT to create interview questions, you must realize that candidates will also use the tool for researching answers. One reporter even used the tool to create cover letters to apply for open positions.
ChatGPT and Coding Assessments
One area that FloCareer has analyzed is the use of ChatGPT to answer coding questions. We've found that ChatGPT generates properly formatted and relevant code. Often times, it may not be accurate which is why it was banned by Stack Overflow. According to Stack Overflow, "trust is broken when users copy and paste information into answers without validating that the answer provided by GPT is correct, ensuring that the sources used in the answer are properly cited (a service GPT does not provide), and verifying that the answer provided by GPT clearly and concisely answers the question asked."
But code created by ChatGPT can certainly be submitted in a coding assessment. In fact, we believe that it may threaten the credibility of coding assessments. We posed the question to ChatGPT.
How can ChatGPT be used to cheat on technical assessments?
It is not appropriate to use chatbots or language models like GPT-3 to cheat on technical assessments. These tools are not designed to replace the knowledge and skills that are required to succeed in a technical assessment. Additionally, using chatbots or language models to deceive others during a technical assessment is unethical and could lead to negative consequences.
Ok, that response is expected and appreciated, but as a follow-up, How can ChatGPT assist in a technical assessment?
It is important to prepare for technical assessments by studying the relevant material and practicing your skills. You should aim to demonstrate your own knowledge and abilities during a technical assessment, rather than relying on outside assistance. Using chatbots or language models in an attempt to deceive others during a technical assessment could be considered cheating and could result in disciplinary action or other negative consequences.
And finally, Would ChatGPT be able to pass a technical assessment?
It is not possible for chatbots or language models like GPT-3 to pass a technical assessment, as they do not have the necessary knowledge or skills to do so. These tools are designed to generate text based on patterns and examples that they have been trained on, but they do not have the ability to understand or apply technical concepts or solve technical problems in the same way that a person can.
While chatbots or language models may be able to provide information or explanations related to technical topics, they are not capable of demonstrating the understanding and expertise that is required to pass a technical assessment. It is important to prepare for technical assessments by studying the relevant material and practicing your skills, rather than relying on chatbots or language models to provide answers or solutions.
Ok, great advice and guidance. And expected, too. However, the fact that ChatGPT has thoughtfully answered these questions suggests that its developers have considered this use case. Given that we believe that candidates will consult with ChatGPT for coding advice and instruction, particularly for help with automated technical assessments or one way videos, how can hiring managers identify its use, mitigate it and ultimately, control it?
Combating the use of ChatGPT in a Technical Assessment
FloCareer has been at the forefront of identifying and combating remote interview fraud. The shift to remote work and remote interviewing has made candidate fraud more prevalent.
The use of ChatGPT as an external reference source is just another tool, albeit a sophisticated one, in the fraud's arsenal. In a one way video assessment where a candidate can stop and start the video or re-record, it may be a challenge to identify the incorporation of ChatGPT generated code from a "scratchpad". Thus, many of the benefits derived from a coding assessment via one way video may be diminished via the use of such a powerful tool.
Live video interviews where you can identify candidates opening tabs via automated tools, as well as physically diverting their eyes from the camera are obvious tells. These cues would be identified and marked as fraud by FloCareer interviewers. In addition, varying the questions and using less obvious skills-based questions as part of the interview structure can diminish the usage and impact of ChatGPT code. Candidate fraud is an on-going evolving approach and the use of tools like ChatGPT is just another technique that can be used to deceive hiring firms. But being vigilant and up to date on the techniques and how to mitigate them is essential.
Conclusions
The usage of ChatGPT has exploded not only in talent acquisition, but most others. Perhaps some companies have over-indexed on ChatGPT and generative AI, but that also means that many will be playing catch up. Specific to the impact that ChatGPT will have on technical assessments and one way videos, it is believed that assistance from the tool will be too tempting for candidates. Thus, diminishing the value derived from these interview tools. Although, we may be biased, we feel that a live video (two way) interview using an interview as a service platform is the way to go.
