Published August 30, 2024 | Version v1
Dataset Open

Using code from ChatGPT: Finding patterns in the developers' interaction with ChatGPT

Description

ChatGPT can advise developers and provide code on how to fix bugs, add new features, refactor, reuse, and secure their code but currently, there is little knowledge about whether the developers trust ChatGPT's responses and actually use the provided code. In this context, this study aims to identify patterns that describe the interaction of developers with ChatGPT with respect to the characteristics of the prompts and the actual use of the provided code by the developer. We performed a case study on 267,098 lines of code provided by ChatGPT related to commits, pull requests, files of code, and discussions between ChatGPT and developers. Our findings show that developers are more likely to integrate the given code snapshot in their code base when they have provided information to ChatGPT through several rounds of brief prompts that include problem-related specific words instead of using large textual or code prompts. Results also highlight the ability of ChatGPT to handle efficiently different types of problems across different programming languages.

Files

1064-CameraReady.pdf

Files (3.4 MB)

Name Size Download all
Checksum: md5:b8ac0a5d6b737639202f31c4c51b616c

PID: http://hdl.handle.net/11304/c94bc0dc-a912-4b2a-973f-f9fbce77f03d
785.0 kB Preview Download
Checksum: md5:4ccd964f6f4c9c8970a520198f8d134d

PID: http://hdl.handle.net/11304/2522a871-68a3-42fa-b8eb-43e8e421908f
340.6 kB Download
Checksum: md5:bfaad5251aaf943c573bf86626541876

PID: http://hdl.handle.net/11304/228a9bc7-e573-4669-9b50-995f9d9b4071
335.1 kB Download
Checksum: md5:4de6df0d681cff785cbbac411d33dbe0

PID: http://hdl.handle.net/11304/8f728851-a6e0-44a8-aa9f-722e94370d90
257.7 kB Preview Download
Checksum: md5:ac7916b4f63744fcf58d18e73cfa7a0c

PID: http://hdl.handle.net/11304/c795f2df-147d-4436-ba83-09cfffe6c3c8
1.2 MB Download
Checksum: md5:5e0497f1cc6f4818791f98393dfcd57e

PID: http://hdl.handle.net/11304/7612ac59-ec87-4458-8a60-421a56433c3d
412.7 kB Preview Download
Checksum: md5:af9d777fd6e58cb2d1058a0747f25f6d

PID: http://hdl.handle.net/11304/35fb6163-d2e4-4c6b-b29b-5356ed346f69
1.8 kB Download
Checksum: md5:519d1e9b8d0af1130e73549ac620a98d

PID: http://hdl.handle.net/11304/03ceeb56-3422-4c34-adb6-e034afa9f50b
1.8 kB Download
Checksum: md5:79815b1fdfbcfffa074c5b282e65c986

PID: http://hdl.handle.net/11304/bacf4b98-53c9-41ec-a83d-f151839197bc
2.3 kB Download
Checksum: md5:18acad0c8c703594b0425f6fbd20b86b

PID: http://hdl.handle.net/11304/a048389f-5e1c-4e60-9dfa-573c580ca05d
7.3 kB Download
Checksum: md5:488cb48326aa2a95d2ff613e562f54e1

PID: http://hdl.handle.net/11304/1c7e8683-e180-4284-86e3-91797d200eee
7.3 kB Download

Additional details

Identifiers

b2rec
db2ef5890fa44c7a85af366a50de73b9