|
Jimer's Guide to Bug Reporting
By: JimerLins
This document describes how to file a good bug report, what a bug report should contain, and provides some examples of good and bad bug reports. For some people, reading this will result in a "Thank you, Captain Obvious!" moment. =) Many people already file good, solid bug reports. Many do not, and not because they're not capable, but because they just aren't aware of what a good bug report is and why. This guide attempts to address some of that.
BUG REPORTS
Bugs. We've all seen them, had them affect us in various ways. Some are funny, some are annoying, some are weird, and some are "game-breaking".
According to the Jargon File, a bug is defined as follows:
bug: n. An unwanted and unintended property of a program or piece of hardware, esp. one that causes it to malfunction. Antonym of feature. Examples: "There's a bug in the editor: it writes things out backwards." "The system crashed because of a hardware bug." "Fred is a winner, but he has a few bugs" (i.e., Fred is a good guy, but he has a few personality problems).
As a software developer, I've dealt with my share of bugs. I've created more than a few, and fixed more than I can possibly count. When users report bugs, developers often get frustrated because the user doesn't always realize what the developer needs in order to find and fix the bug, and so the bug report is incomplete or not useful. This means the bug probably won't get fixed or won't get fixed to the user's satisfaction.
Then the user is left wondering why the developer didn't fix the problem. After all, he reported it- why didn't it get fixed? Bugs are going to exist in any software; that's a given. The more complicated the software, the more bugs are going to be in it, in general. The biggest problem with diagnosing/fixing bugs is that quite often fixing one bug can create others- this is called the "law of unintended consequences" and is the bane of software developers.
In a game like SWG, there are probably dozens, if not more, developers. They also have a very large test team, if reading the back of my game manual is any indication. Your bug reports go to some group inside SOE; it is *highly* unlikely that they go straight to a given developer, who has a large list of things on his or her "to-do" list. What is more likely is that they go
to a tester for checking and reproducing, then entered/updated in SOE's internal bug-tracking system.
What does this mean and why am I explaining this? Because it means that when you file a bug report, you're not talking to the developer who wrote the code with the bug in it. You're talking to a tester, most likely, who knows as much about the actual code as you do- that is, not a thing. Testers aren't developers; it's important to communicate as much as possible about a
given flaw in the system to them as is humanly possible so that they can find a way to reproduce the issue and get it sent on to the dev team for fixes.
THE BUG REPORT INTERFACE
In the game, if you run across a bug, you can start the bug report interface in one of two ways: Press CTRL-H and click the appropriate button, or just type /bug. Do not confuse bug reports with CSR tickets! A CSR ticket is a request for a CSR (Customer Service Representative) to assist you with something in the game. If you report a bug using the CSR ticket request, it probably won't get filed properly and you'll waste the CSR's time. If you file a request for assistance using the bug reporting interface, you probably won't ever get any assistance and your issue will remain unresolved.
CSR tickets are to correct things like being stuck in a wall or in the air, or when you have immediate problems that may be caused by a bug, but require an administrator's assistance to correct. CSR tickets are not for reporting bugs! If you run into a bug that also causes you to need a CSR, file a ticket to get help and file a bug report while you're waiting for the CSR to get back to you.
The bug reporting window has a number of options, like how severe the bug is, whether or not you can reproduce it, what game system it affects, and so on. Set these options as best you can; I've quite often found that many situations aren't covered by the items available on the menu, so do your best when that occurs. The most important part of a bug report is the description you provide, not the menu items you select.
Note: I'm not getting into the specifics of the /bug interface. It's confusing, and much of the information you can provide is either redundant or not always applicable. My recommendation is to fill out the form as best as you can, then put all the details you can provide in the actual text description of the bug, which is far and away the most important part of the bug report.
GENERAL DO'S AND DON'T'S
DO:
- Be polite.
- Include as much information as you can about the bug and the circumstances surrounding it.
- Use the English language (except obviously for those playing in other countries with a different language) and type out all your words. If you need a spellchecker, use an offline editor like Notepad and paste the bug report into the bug interface.
DON'T:
- Be rude, inflammatory, or nasty.
- Demand that the problem be fixed.
- Threaten to quit the game (if you're doing that, you really should just quit now).
- Use "l33t" or shortened words- "4" instead of "for", "u" instead of "you", "ur" instead of "your"/"you're". Just. Don't. Do. This. Seriously.
THE COMPONENTS OF A GOOD BUG REPORT
Every system is different, and the requirements for reporting bugs on those systems will be different. Here's my assessment of what's needed for a good bug report in SWG; the five "W"s: Who, What, When, Where, and Why, along with Repro Steps (more on this later).
Who Are You?
Your character name is important. Even though you're filing this bug in-game, which means your character and account are attached to it, include your character name in the bug report. If you have multiple characters, be sure you mention the one that you encountered the bug on! Also, which Galaxy are you on? Include that as well.
What Happened?
Describe, in detail, the bug. "In detail" means IN DETAIL. Provide all the info you possibly can, even if you think it's not necessary or the developers already know. You would be amazed at how often it's a tiny, totally unrelated detail that actually provides the answer or a hint to the developer as to what the problem might be. In summary: More Information Is Good (tm).
When Did it Happen?
Even if you're filing the bug report at the time or right after you ran across a bug, include the date and time, make it down to the minute if you can- and include your time zone, please! "Around 11 PM last night" is useless to someone reviewing the bug report two days later in a different time zone; they've probably got about a thousand more to get through, and I can promise you the ones that require additional work to figure out exactly when it happened are not going to get the attention. Time zone is important, because this game is played by people all over the world, and the developers probably don't live in
your time zone. "12/05/2004 at 10:53 PM PST" is much, much more useful.
Remember also that quite often, the process of hunting down bugs means going through system logs, which on this game, are probably quite large. So if the best you can give the poor sod that's trying to figure out what happened is a 30-minute window, that's a lot of logs to go through. If you can narrow that down to a one or two-minute window (or even 5-10 minutes), you've just saved the person hunting the bug a lot of work.
Where Did it Happen?
Be as specific as possible, and include the planet and waypoint if you can. If it's something that's happening on an entire planet, say so, but mention where you first encountered the problem if applicable. Remember- more detail cannot hurt, and just might be the thing that solves the puzzle. If it happened in space, be sure to include the system and the full coordinates (there's three for space waypoints!) of where it happened.
Don't say things like "My house" without saying where your house is. Why make them figure out where your house is located? And if it did happen in your house, tell them which room!
Why Did it Happen?
This one's tricky- you may not always know exactly why a bug happened to you. But this is where you get to describe what you were doing or what was going on when the bug was found. But provide as much detail as you can- include what you were doing, what you had done recently, and any other details you can come up with- even what you were wearing, if you were buffed, what food you'd eaten, whether you had wounds or BF, etc. Remember, even seemingly unrelated details can be important when
tracking down what caused the bug.
Repro Steps
This, to a developer, is arguably the most important part of a bug report. How do you make it happen? Unfortunately, we can't always reliably reproduce a bug. In other cases, we could reproduce it reliably but don't want to (imagine a crafting bug with RIS Armor; I'm not gonna lose a bunch of GDK scales just to prove I can make it happen more than once! ).
When you do know what is required to make the bug happen, verify (if possible) that you can reliably reproduce it, and then put the steps that are required, in order, in your bug report. Be sure to include any basic assumptions that may not be implied by the steps- if the bug occurs only when you're wearing a particular piece of clothing, be sure to include that.
Don't be afraid to be pedantic here; making the steps as clear and concise as possible will help and can't hurt. Not every person that sees your bug report will be a developer, and not every developer that does see it will be familiar with every system in the game. Also, it is quite likely that internal testers receive the bug reports and need to validate them in some manner before they get to the dev team. That means that if you can make sure someone can repro your bug by simply following directions, your bug report gets handled sooner.
BUG REPORT TEMPLATE
Here's a basic layout for submitting bug reports. This isn't required or anything, but it does provide a nice framework for getting all the relevant bits in one place. If you're submitting a bug report and something isn't applicable, then just leave that part out. For example, a bug report about a word being misspelled on the Character Skills screen probably doesn't require as much detail as one that involves a complex combat or crafting bug.
---------------------------[Cut Here]-----------------------------------
Character Name: [Your character name here. This is NOT your login name]
Galaxy: [Your Galaxy or Galaxies if more than one]
Summary: [A concise description, in one sentence, of the bug.]
Date/Time: [Date and time you last experienced the bug.]
Location: [Full coordinates, if possible]
Description: [Include a detailed description of the bug, what causes it or seems to, and any other information you can think of that might be useful to someone trying to diagnose the bug]
Repro Steps: [Step-by-step instructions, if possible, on how to make this bug occur. If you can't provide full repro steps, note this and provide whatever you can. Even small amounts of information can help.]
---------------------------[Cut Here]-----------------------------------
EXAMPLES
Here's an example of a good bug report. No, this isn't a real bug. I made it up. Don't Panic. This would be the
portion of the bug report I would place in the text box of the bug report interface.
I have included the text "[THIS IS A SAMPLE, NOT A REAL BUG.]" several times so that people searching for bugs don't think this is a real bug.
---------------------------[EXAMPLE]-----------------------------------
Character Name: Jimer Lins
Galaxy: Scylla
Summary: Schematics made with with exactly 100 items cause factories to disappear (this is an example, not a real bug.)
Date/Time: December 13, 2004 8:53 PM Pacific Time
Location: Lok, -3223 1293
Description: If I make a schematic for any item I can craft and put 100 items as the manufacturing limit, when I run the schematic and the factory is done, instead of simply shutting down, my factory vanishes.
I have verified that this occurs with any item I can make. If I make it for less than 100 or more than 100, the factory operates normally and no problems occur. Also, if I put more resources than is required to make 100 items, a schematic with exactly 100 items on its limit works normally.
In summary, this occurs when you make a schematic for exactly 100 items, no more and no less, and include precisely the amount of resources required to produce the item in the factory's ingredient hopper. [THIS IS A SAMPLE, NOT A REAL BUG.]
Repro Steps: (assumes you're a Master Artisan, and using a Weapon/Droid/General Crafting tool. I use a +42.15 crafting station, located at the coordinates above.)
- Craft a schematic for any item (say, a dice set).
- Set the manufacturing limit to exactly 100 on the schematic.
- Use the schematic in an Equipment Factory. (in my case, the factory had about 10 days' worth of maintenance and 7 days worth of power)
- Place exactly the amount of resources required in the ingredient hopper to produce the item. Using more or less than is required will not reproduce the bug.
- When the schematic is completed, the factory will vanish. [THIS IS A SAMPLE, NOT A REAL BUG.]
---------------------------[EXAMPLE]-----------------------------------
This bug report provides all the necessary information for a tester or developer to diagnose the bug, or at the very
least provides the info necessary for them to start digging.
What it does NOT do is take out the consequences of the bug (losing a factory and all the stuff in it) on the hapless tester or dev that gets to pull this bug out of the queue. There's no ranting, no demands to fix the problem right now, and no demands to return the items lost. Remember, bug reports are NOT CSR tickets. You won't get any customer support for a bug report and you won't get a bug filed when you put in a CSR ticket.
If the situation described above were real, I'd also have filed a CSR ticket (maybe even including the bug report ID, which is sent to you when you create it) about the situation. I'd at least be likely to get my factory back.
Here's an example of a bad bug report.
---------------------------[EXAMPLE]-----------------------------------
DAMMIT SOE YOU Stink! I wuz makng tools 4 my guild n my factory disappeard!
FIX IT NOW! I WANT MY FACTORY BAK! YOU SUCK!!!!!
If u dno't fix tihs n give me my factory bak, I'm gunna quit this stupid game!!!
---------------------------[EXAMPLE]-----------------------------------
I don't think a description of exactly why this is a bad bug report is required. However, I can *promise* you that the SWG
team gets a lot of these. I also doubt that any bug report that even remotely resembles this one is going to be looked at.
There's no relevant information, and frankly- if I was the one reading this bug report, I'd assume that the person who wrote it didn't
pay their maintenance and not give it another thought.
Remember, you're dealing with human beings, and you may actually know some game systems better than the person who reads your bug report. That's NOT because they're incompetent or because you're an uber gamer gawd, it's because this game is huge and not everyone on a test/dev team will know every aspect of the game, any more than you do.
SUMMARY AND CONCLUSIONS
Basically, I hope that this guide gives people a good starting point for filing solid bug reports that the testers and dev team can work with. I'll be happy to incorporate any constructive feedback into revisions of this guide.
|
