Anonymous messages in Discord using Google Forms
Published 4/22/2022 | 1,372 Words
This article is part of a series on The Development of Kevin.
One of the unique challenges in developing a piece of software designed to satisfy the needs of a single community is that as the needs of that community change, the software needs to adapt as well. These changes in needs can sometimes happen quickly, and with Kevin specifically, it was expected that Kevin would implement new features very soon after they were conceptualized. This often led to feature implementations that were quick to get working, but were not the best solution to the problem at hand or had significant problems due to lack of careful testing.
Anonymous forms are one such example of a feature that was designed quickly and then iterated upon afterwards. As such, there are two implementations of anonymous messages to talk about in this article; the first implementation which was hacked together very quick, and the second improved implementation put together months later when the limitations and issues with the first implementation had become apparent.
Implementation 1
The first implementation of anonymous messages was created when an "Anonymous Concern" form was implemented for the community. This form was created in Google Forms and allowed members of the community to send in concerns that they had about other members or the community at large to the community staff. When this form was created, it was apparent that given how pressing these concerns often were, it was not an adequate solution to have someone check the form every day. There needed to be a quicker way for staff to be notified once a concern came in.
To this end, a system was implemented in which Kevin would authenticate using a service Google account using google-spreadsheet and would read from the form response Google Spreadsheet that was updated whenever a new concern was submitted. Kevin would grab this spreadsheet every 5 minutes and loop through every row in the sheet and check if a checkbox in the "reported" field had a value of TRUE. If the value was false, Kevin would send an embed containing information about the unreported concern into a staff-only channel for staff review.
let channel = bot.channels.cache.get('000000000000000')
await anonConcerns.useServiceAccountAuth(require('./credentials.json'); //Authenticating with Sheets
let msg = new Discord.MessageEmbed;
await anonConcerns.loadInfo(); //Loading and creating an array of sheet rows
let memberListReal = anonConcerns.sheetsByIndex[0];
let rows = await memberListReal.getRows();
for (i=0; i < rows.length; i++ ) { //Checking for unreported concerns
if (rows[i].reported === 'FALSE' || rows[i].reported == null || rows[i].reported == '') {
msg.title = ('New Concern - ' + rows[i].Timestamp);
let desc = rows[i]['What is your concern?'];
if (desc.length > 1200) { //Trimming long concerns in order to avoid an invalid API request
let contentTrim = desc.substring(0, Math.min(desc.length, 1200));;
contentTrim += '...';
msg.description = contentTrim;
} else {
msg.description = desc;
}
channel.send(msg);
rows[i].reported = 'TRUE'; //Setting the reported field to 'TRUE' and then saving these changes to the Google Sheet.
rows[i].save();
}
}This system had numerous issues endemic to its design. Starting with the authentication with Sheets and loading the actual sheet, it was very often that authentication would fail, and even if authentication succeeded, loading the sheet would often fail as well. Whether this was due to an issue with google-spreadsheet or some API limit that was not made obvious to me, these rampant errors often made a theoretical 5 minute interval between searches closer to a 15 or 20 minute interval. That was fast enough in most cases, but in cases especially in which staff was trying to communicate anonymously with the concern submitter, the increased delay caused headaches and often resulted in me initiating a scan manually by restarting the entire Kevin process.
Additionally, when the system was first implemented, the description trimming code was not yet written, and there were a couple occasions in which an error would be thrown due to someone submitting a concern over 2048 characters, which would cause Discord to reject the sending of the embed. This resulted in staff never receiving the message since the concern would then be updated to a reported status, though the embed was never sent. Thankfully, there were no instances in which a concern rejected by the API was of urgent importance.
The last glaring issue with this system of reporting anonymous messages was that occasionally, even if the embed containing the message was sent, an error would occur when updating the status of the message to reported. This would cause an embed to be sent, and then the same embed to be sent 5-20 minutes later, depending on how many errors were encountered while trying to authenticate and retrieve the sheet.
It became clear after a while that as these anonymous messaging forms were being used more often, the need for a new system of reporting these messages was needed.
Implementation 2
There was about a year between the first implementation of anonymous messages and the second implementation. During that period, Discord released some new exciting features for bots. Specifically, slash commands and buttons were making interacting with bots easier than ever before.
I wanted to make extensive use of these features for Kevin, and that required me to learn about web servers and interacting with the Discord API without a library as discord.js had not yet been updated to support these new features. This newly acquired knowledge would turn out to be of great use in creating an improved system for anonymous messages.
The basic flow of the second system for anonymous messages that require approval is outlined below:
The first and most consequential change made with this system is that the way Kevin gets the signal to output an embed into a staff channel for approval is by using an HTTP request. This HTTP request is sent using Google Apps Script UrlFetchApp functionality. This UrlFetchApp code is placed inside of a function that runs whenever someone submits a message using the form. An authentication key is defined above the function in order to prevent API requests not originating from the Apps Script.
function anonVent(e) { //e == information about the submitted message
var headers = {
"Authorization" : "Basic " + key
};
var options = { //Configure the api request
"method":"post",
"contentType":"application/json",
"payload":JSON.stringify(e.namedValues),
"headers":headers
};
UrlFetchApp.fetch('https://api.gtv.lgbt/spreadsheet/anonvent', options);
}The function run when a new message is submitted
The request sent by the Apps Script is then received by a component of kevin called kev-interaction. Kev-interaction is an express.js web server that runs all the interaction functionality for kevin as well as anonymous messages. Once the kev-interaction process receives the request, it turns the data contained within the request into an embed with approve/disapprove buttons and sends it to a staff-only channel.
async function anonVentPost(details) {
client.api.channels('').messages.post({
data: {
embed: {
title:`New ${details['Is this a vent or a confession?'][0]} - ${details.Timestamp[0]}`,
fields:[
{
name:"Vent",
value:details['What would you like to say?'][0]
}
]
},
components: [{
type: 1,
components: [{
type: 2,
label: "Approve",
style: 3,
custom_id: "anon_vent_accept"
},{
type: 2,
label: "Deny",
style: 4,
custom_id: "anon_vent_deny"
}]
}]
}
});
}API Request for embed sent when an anonymous vent is submitted
Using an HTTP request solved the problems with reliability and consistency experienced under the old system. Now, instead of having to authenticate with Sheets and scan the sheet every 5 minutes, the reporting of anonymous messages was now instantaneous. The new system at the time of writing has only encountered one error throughout its deployment, and that was due to an incorrect filling out of the form itself and not a problem with the code.
This system also moved away from using reactions for approving messages in favor of buttons. Buttons are not only larger and clearer; they also do not require the main kevin process to always have the embed message cached in order to wait for a reaction. This change made it so that if the kevin process restarted, the message could still be approved and would not have to be manually typed in the public channel by a staff member.
await client.api.channels('0000000000').messages.post({
data: {
embed: {
title:`Anonymous${(message.embeds[0].title.replace(/\-(.*)/,'')).replace(/New/,'')}`,
fields:message.embeds[0].fields
}
}
});API request made when the Approve button is pressed
These new approaches to reporting and reviewing messages allowed the system to be expanded from just the two forms we had using the old system--anonymous concerns and anonymous vents--to a larger set of anonymous messages. It did not require us to change the way users submitted these messages, and the addition of buttons allowed for a better experience for staff members.
More broadly though, the differences between the two systems are a reflection of the development cycle that many of the features included in Kevin went through. The pattern of quickly making a feature to respond to the needs of the community and then refining the feature later is repeated many times across kevin. Often times these refinements were not immediately apparent to members of the community, but they were as important to the development of kevin as any shiny new feature.
It is my sense from other teams that often because these refinements are not as eye-catching or exciting, they are ignored. As a result, old or inefficient systems and content are left in place to diminish the quality of anything built on top of them. Thus, one of the lessons I have taken away from kevin is that sometimes the most productive use of your time is not building the next shiny and new feature. Instead, fixing the 'weakest link' in a piece of software can save you more time and effort later than the time and effort you put in to fix it.