Author: Chris J

When processing data, the foremost and maybe even the central choice one needs to make for data entry and management is the choice of utilization of the flat database vs. the relational database. Both of these options have pros and cons associated with them, which in turn grow or lessen with respect to how you intend to use your data.

However, first let’s provide some definitions: Flat databases are built from a single file, a single table. Relational databases are built from multiple files or tables and relate to each other via special key fields – hence the term, “relational.”

Flat Databases:

Pros:
Extremely easy to start using: Anyone with access to a computer is likely to have some variety of spreadsheet programs, be it Excel for Windows users or Numbers for Mac OS. Even if both of these options are unavailable, Google Sheets is available free for everyone with a Google account. All of these options can be almost immediately opened upon starting up one’s computer, allowing data entry to start promptly.

User-Friendly: In all the options listed above, as soon as the program has started a table is easily constructed with data entry being self-explanatory. New headers and columns can be easily added and manipulated in these programs, making the initial setup a breeze. For relatively little work you have a table that is easily readable and cohesive.

Cons:
Bad handling large amounts of data: Most of the problems with flat data emerge when the spreadsheet grows larger. I estimate about 200+ entries would be the cutoff point for when one should begin thinking about switching to another method, (can’t imagine that it would work above 1000 entries) – see why below.

Starts looking sloppy: If you go flat, the more data you enter into the spreadsheet, the more variation and duplicity you are likely to see in this data. Therefore, in your spreadsheet, you are going to see columns of different lengths and widths. Various categories for each data point also complicates spreadsheet filing. Sloppiness may lead to increased error rates.

Rigid: Flat data and spreadsheets often are quite rigid and time-consuming to manipulate; especially the more information entered. If a mistake is found later in the data entry processes or if one wishes to add or remove a category, complications can emerge.

Relational Databases:

Pros:
Over the Internet: Most Relational Databases will normally at minimum have internet compatibility or exist as a web program. It is also possible to establish permissions rights for various parts of the data, such as viewing only or editing only of particular locations.

This option allows for easy collaboration or sharing of data.

Logical categories/subcategories:  There is a large amount of compartmentalization with each value existing under a new header, which itself is in under another larger parent category, keeping all the information organized.

Easy manipulation and search-ability: Once all the data has been successfully entered into the relational database, manipulation becomes a breeze. Because almost every value is separately compartmentalized, it is extremely easy to find the value or group of values one is searching for and then to move or reposition them in a new location.

Cons:
Harder to get working and somewhat of a general pain: When the program is first entered, it’s a pain to start designing all the separate headers and categories under which to store the information. I attempted to recreate the system described in Stephen Ramsey’s article. However initially learning to use the program proved difficult not to mention that if one were to do it from scratch, a clear picture of the goals is necessary

Conclusive Thoughts: When choosing a database format, the most important thing to think about is its purpose. If you are collecting large amounts of data under various categories and groupings, hoping to then compile all the information, then a flat spreadsheet is not the way to go. Relational database utilization might provide you the “bang for the buck.” However, if the project is a quick analysis of 200-300 points with under 5 groupings, then flat data will probably save you a lot of time.

Other Issues to keep an eye out for:

Plan – Know what you want. Know where you’re going: It is always good to have a rough outline planned for how much data you are going to collect and your plans for processing the information before you get started. This planning can help you avoid problems down the road, such as selecting the wrong database format.

What are the Categories?: Know the categories and groups under which you sort the information before you begin entering it. This preparation will allow for less confusion and avoid redundancy.

Consistency is Key: Be consistent in how your data is entered and in your project as a whole. The more consistent you can be, the cleaner the final product; the nicer the information you can draw from it.

I was unsure about where to post the newest blog, so I ended up posting it on the new web pages we had created. It can be seen Here.

Sorry about the inconvenience.

The digital project that I chose to reverse engineer was the Mapping Occupation project By Gregory Downs and Scott Nesbit. This project in the Digital Humanities,

“captures the regions where the United States Army could effectively act as an occupying force in the Reconstruction South… it presents the basic nuts-and-bolts facts about the Army’s presence, movements that are central to understanding the occupation of the South.”(1)

When entering the website, one is faced with a large map on the right and a navigational/information bar on the left. There are two options for exploring the data and information presented. The first method is exploratory, allowing the viewer to freely zoom in and out on the large exploratory map that fills 80% of the screen, adjusting displayed information relative to date. Information that can be adjusted includes total troops, cavalry, black troops, zones of access, occupation to railroads, and voting data. This approach allows for a free examination of the data. The second approach is more structured than the first, featuring a narrative map through the site. This approach tells a story of army occupation from near the end of the war to past the 1870’s, allowing for an analysis of the data, drawing on the components of politics and culture to add contrast.

When the website was describing how its data and analysis were conducted, there seem to have been two inquiries. The first was data collection information that would form the backbone of the website such as location, number and time that US troops occupied certain areas. These data were collected via the final monthly military reports made by the military. When information was unavailable for a particular month, data from other reports were made in the particular time period; conflicts were resolved by use of the most detailed report. Likely sources for error and a more detailed decryption of data compilation are available on the website.

The second inquiry was a little more complex to conduct as it involved defining the likely zones of occupation and access. Extrapolating from the data given via the troop’s station, these zones were calculated. Occupation zones were found by using a distance that a unit Calvary or infantry could travel in a day’s walk or 6-hour train ride. Zones of access were formed by “places from which freedmen might reasonably travel to a military outpost.”(2) These, however, were more constrained as the former slaves could only travel by foot without owning houses and train access being restrained. All the data are available for download on the website.

My main annoyance with the website is its separation of numerical data and narrative data. The Map does a nice job featuring the large amounts numerical data visually; however, I find myself getting annoyed when attempting to learn about a specific location. When I try to click on particular troop stations, it would be functional and user-friendly if an info window would appear to provide troop numbers and maybe even a short blurb on the current situation. For example, what’s that one unit of some 5,000+ men doing in the middle of the Ocean in the gulf of Guinea? Instead, if I wish to discover more on the narrative front I need to read through the long paragraphs featuring white text on a black backroad, making it difficult to read extended time on a computer. This issue brings me to my secondary annoyance: the black and white color scheme becomes difficult to navigate after a few minutes. A white/grayscale with sparing amounts of black would have been more appealing.

In summary, I found the website informative and can see its value under the right circumstances, say a civil war analysis or for information on racial tensions in the post-war US. The website could use some fine-tuning in map navigation and interface, however, I’m just nitpicking, in the end, it provided an interesting interface trough which to study the US troop occupation of the south.