Homework 7 - Murder mystery (due Friday 11/18, 11:59:59 AM)

(Credit to Brent Heeringa and Thomas P. Murtagh of Williams College for the original steganography homework concept and TAs Sam Shuster, Spencer Lee, and Alice Yang for coming up with the plot!)

Saturday, November 5th, 2011:'Remember remember, the fifth of November'

Peter-Michael is dead. Killed in his own house. Body found in home study.

Cause of Death: A decaying, half-eaten mango was found near the corpse. Forensic testing showed the mango had been laced with lethal quantities of arsenic. An open window on the computer showing a terrible defeat in Starcraft and concomitant massive loss of Karma and 'cred' point to a suicide. However various documents and images retrieved from his hard drive indicate foul play and the possibility that Peter-Michael knew the identity of his murderer.

Sunday, November 6th, 2011 — Though investigation is still in progress, the investigative committee has made some headway.

  1. Peter-Michael had been an under cover investigator for the Presidential Security Group, the secret service of the Philippines.
  2. It has recently been discovered that the Mango Haters of America (MHOA) has been targeting the mango industry in the Philippines for almost a century. This elite group of terrorists is rumored to be based in Philadelphia. Ever since the decline of the exotic fruit industry in the region, Philadelphia has been known to be the center for most fruit-hating organizations around the globe. Recently, the MHOA have been rumored to be close to finishing a devastating plot against the mango producers in the Philippines, a move could cripple the world economy. This shadowy organization is said to be headquartered at the University of Pennsylvania, the cultural epicenter of Philadelphia.
  3. Peter-Michael, the supposed PhD student, had made significant progress into his investigation. Chat logs found on his computer hint toward an impending significant discovery. The logs also seem to indicate an almost irrational paranoia of his inner circle.
  4. Video footage captured by Peter-Michael's webcam shows the final moments before his death. It appears that he was aware of his death and took his own life before MHOA tried to capture him. Based on the video, the committee believes that Peter-Michael managed to hide the evidence against MHOA somewhere on his computer or perhaps distributed it over the Internet in some encrypted fashion.

Monday, November 7th, 2011 — After being notified of his demise, Peter-Michael's twin brother, the famed ethnomusicologist and playboy Michael-Peter, immediately flew to Philadelphia. He was quoted as saying, "Well that's more terrible than a spoiled mango. Although I am not as computer savvy as my brother Peter, I know he will have imparted enough of his unspeakable knowledge to his students in CIS 110. I have read through his lecture notes and am more than ready to teach the class."

The Committee hopes this is true.

A new hope

The Committee will continue this investigation, but is comprised largely of old men and puppies. As such, the committee has virtually no hope of decoding the informaton found on the computer. It is up to you, the members of CIS 110, to decode Peter-Michael's dying message to the world, stop the Mango Haters of America, and avenge Peter-Michael's untimely demise.

To provide further incentive, the Committee has decreed that this secret mission, the success of which will determine the fate of our world, shall be part of your overall homework grade, as outlined in your course syllabus. Peter-Michael was fond of Steganography, the art and science of writing hidden messages, so the committee thinks it is likely he used digital steganography techniques to hide this information from his assailants. While the CSIs continue to extract data from Peter-Michael's hard drive, your task is to write a pair of programs that inserts and extracts hidden text from images in anticipation of any data that might come out of the investigation.

Steganography with image files

In computer science, we can employ the art of bit stealing in order to hide messages in digital artifacts such as image files.

Bit stealing

Suppose we have a set of sample data, e.g., prices of houses in the Philadelphia area:

530211  275358  724121  315210  245884

It's typically silly to talk about the costs of houses right down to the dollar since the lowest digits are insignificant. This is because the numbers themselves are so large and also because it is expected that the lower-order dollar amount of the houses will change rapidly over time.

Because the lower order digits are essentially unused, if we are crafty, we can steal them to encode additional, hidden information. For example, consider a slight variation on the original data set.

530211  275359  724121  315210  245884

The only differences between this data set and the previous one is that the ones digits have been replaced. In fact, they have been replaced with the University of Pennsylvania's zipcode, 19104. If I wanted to communicate UPenn's zipcode secretly (e.g., to signify to my friends to go there for a meeting), I might broadcast these subtlely different housing prices to a public forum with the expectation that people "in the know" will know to look at the last digits of the prices for the location. Everyone one else will be oblivious because the ones digit we used is otherwise insignificant.

How might we do this programmatically? We learned earlier that computing n % 10 will produce the ones digit of n and doing integer division n / 10 will "delete" the ones digit of n. With this in mind, we can do three useful operations:

  1. Clearing: given a number n clear its lowest-order digit so that it is always zero.
  2. Insertion: given a number n, clear its lowest-order digit and insert some other digit in its place.
  3. Extraction: given a number n, extract out the lowest-order digit.

In the context of the above example, implementing the three operations is straightforward:

  1. Clearing: (n / 10) * 10 produces n but with the ones digit cleared.
  2. Insertion: given that n has already been cleared, we can compute n + k where 0 <= k <= 9 to insert k into the ones digit of n.
  3. Extraction: n % 10 produces the ones digit of n.

Note that this technique of bit stealing can be used in any context where we can "get away with" stealing a small chunk out of each datum. In particular, we can use this technique to steal a bit from each pixel in an image to encode some data!

Pixels, images, and colors

As we learned in chapter 3, digital images are made up pixels. Each pixel represents a sample or small piece of the overall image. In this respect, we can think of an an image as a two-dimensional array where an individual element of the array represents a single pixel of the image.

The elements of this array contain the color that should be rendered on the screen for that pixel. There are many color models that are used to represent a color in terms of a number. For our purposes, let's assume that the pixels are given in the RGB color model. If you recall our discussion of the Color class during chapter 3, we are able to make new Colors by calling new Color(r, g, b) where r, g, b are integers in the range 0-255 and represent the redness, blueness, and greeness of that Color.

The RGB color model is precisely this setup where we have three different color components for each pixel. However, instead of using three different integers for each pixel, we use a single integer value and simply pack the three values into this integer. Note that this is possible because an integer is 32-bits total and to represent each component we only need 8 bits for each (since 2^8 = 256). These are exactly HTML color codes if you are familiar with HTML. For example, the hexadecimal number FF0000 can be broken down into its components

  FF      00       00
|-Red-||-Green-||-Blue-|

so that the red component has value 0xFF = 255, and green and blue have values 0x00 = 0.

Stealing pixel images

To steal bits from a pixel value, rather than clearing out the lowest-order decimal digit by dividing and multiplying by 10, let's instead clear out the lowest-order bit by (integer) dividing and multiplying by 2.

To see how this works, let's consider carrying out these operations on a smaller number, say 25. In binary, 25 is 11001. 25 / 2 = 12 which is 01100. 12 * 2 = 24 which is 11000. If we compare these numbers side-by-side:

      11001
/ 2 = 01100
* 2 = 11000

Dividing by two has the effect of shifting the values of the bits right one position, filling in the vacant left-most bit with a 0. Multiplying by two has the effect of shifting the values of the bits left one position, filling in the vacant right-most bit with a 0. These bitwise operations actually exist in Java in more general forms, but we'll stick with describing the bit operations we need in terms of divide, multiply, and mod for consistency's sake.

So we know how to steal a single bit from a number. However, does it make sense to try to steal bits from pixel data? Consider the color 0x007F47 which is a dark green. And now, let's clear its lower-order bit as described above. The resulting color 0x007F46 is displayed side-by-side with the original color below. Can you tell the difference?


0x007F47 and 0x007F46 side-by-side.

Most likely not! The lowest bit of a number contributes only the value 2^0 = 1 to the overall number. This means that bit-clearing will, at-most, only change the blue component of the pixel by one. For practical purposes, this change will be imperceptible to the naked eye.

Because of this, we can steal the lowest-order bit of each pixel in a digital image to store some information and have confidence that people won't know it's there!

Encoding text as a sequence of bits

Now that we've established that we can steal bits from images, what can we store with those bits? Recall that each character in Java is really a number, the character's associated character code. As long as we stick with English characters (specifically, those that fall under the ASCII character set), these character values can be encoded using eight bits. Since we steal one bit from each pixels, we'll need to use eight pixels for each character we wish to encode.

As a concrete example, let's consider encoding the letter 'Q' into a 4x2 image.

           
           

The character value of 'Q' is 81 (as per this ASCII table). The binary representation of 81 is 01010001. Let's suppose our convention is that we layout the bits of the character in the image from left-to-right top-to-bottom. Then we should encode the bits as follows starting with the least-significant bit (right-most) to the most-significant bit (left-most).

1 0 0 0
1 0 1 0

Note that this flips the order of the bits around relative to how we would write them down on paper. We could encode the binary number in either order (this is called the endianness of a value). It turns out that laying out the bits like this with the least-sigifnificant bit first rather than last will make our code simpler to write since we will either generate or need to access the least-significant bit first several times in our program.

The fact that the image is two-dimensional does not affect how we encode a character. For example, we would assign bits to pixels in the same way if the image was 2x4.

1 0
0 0
1 0
1 0

Thus, for our purposes, we might as well consider an image as a one-dimensionsal array where the rows of the image are laid out end-to-end.

1 0 0 0 1 0 1 0

With this representation, our job is much easier. We simply need to read the pixel data array from left-to-right in eight byte chunks to either encode or decode the character data.

Bit-ifying characters

Now that we have established how we'll store characters in the low-order bits of an image's data, the final step is translating from a character value to bits and vice versa.

Character to bits. Given some character value c, we need to extract out the eight bits of that value. This is precisely the extraction operation that we discussed earlier except instead of a decimal digit, we want just a single bit. To do this, rather than calculating c % 10, we need to calculate c % 2. Note this is the same calculation we made to determine if a number was odd or even. If a number is odd, then it's lowest-order bit is 1, otherwise it is 0.

One wrinkle that you may encounter is that Java may report an unexpected result for c % 2 that may throw you off if your test tests explicitly for positive one. Java does this because it keeps the sign of the dividend c. Recall that Java treats the most significant bit of an integer as the sign of the number. The pixel data does not make this distinction, but since we use a Java int to hold these values, then our % results may either be 0, 1, or -1. However, this does not affect our solution in any serious way; simply make sure that your test covers the fact that -1 means there's a 1 in the least-significant bit of c.

In addition to extracting the least significant bit with mod, we also need to remove the least significant bit to get at the remaining bits. Simply dividing the number by two serves this purpose.

For example, imagine that we were turning the character 'Q' into bits to be encoded in an image. 'Q' has character value 81, so let's take successive mods and divisions by 2 to derive the 8 bits of this value:

81 % 2 = 1  81 / 2 = 40
40 % 2 = 0  40 / 2 = 20
20 % 2 = 0  20 / 2 = 10
10 % 2 = 0  10 / 2 = 5
5  % 2 = 1  5  / 2 = 2
2  % 2 = 0  2  / 2 = 1
1  % 2 = 1  1  / 2 = 0
0  % 2 = 0  0  / 2 = 0

Recall that we extract the least-significant bit first, so we read the resulting binary number "bottom-up". Thus our method of succcesive mods and divisions yields 0101001 which is the binary value we previously asserted 'Q' to be.

Bits to character. Given a set of bits, we'll need to calculate the character value from them. In general, in a base-n number system, the contribution of a digit d in the kth place to the overall value of them number is d * n^(k). For example, the value of the decimal number 4158 is:

  4*10^3 + 1*10^2 + 5*10^1 + 8*10^0
= 4000   + 100    + 50     + 8
= 4158

In the case of the binary number above, the calculation would be

  0*2^7 + 1*2^6 + 0*2^5 + 1*2^4 + 0*2^3 + 0*2^2 + 0*2^1 + 1*2^0
= 0     + 64    + 0     + 16    + 0     + 0     + 0     + 1
= 81

Representing bits. In both ImageDecoder and ImageEncoder, you will be working with bits (0-or-1 values) a fair amount. Java does not have an explicit type for a single bit. So you will need to use a number, e.g., int or byte as a bit even though this is overkill. A boolean also works because it represents either true or false that we can use to represent a 1 or a 0. For the purposes of your programs, using any one of these three types is fine as long as you are consistently using that type throughout your programs as a stand-in for a bit.

You may find the ExtractDigit.java program on the lectures page useful to understand for this part. While extract digits works on individual decimal digits rather than bits, the flow and approach are the same.

A complete example

Let's tie all of these concepts together and walkthrough the encoding and decoding process for a small example by hand. Let's use the above example of encoding the letter 'Q' into the 4x2 image which we can consider as a 1-dimensional array of pixel values (where the rows are laid out end-to-end).

#27bc31 #ffae01 #27bc31 #ffae01 #ffae01 #27bc31 #ffae01 #27bc31

The hexadecimal numbers inside now represent the actual color values for those eight pixels.

First, let's convert 'Q' into a binary number. To do so, we convert 'Q' to its character code or value which, according to asciitable.com is 81.

Now, we convert 81 to its binary representation which we do via the method of successive modulos and divisions by two described in the section on turning characters into bits.

81 % 2 = 1  81 / 2 = 40
40 % 2 = 0  40 / 2 = 20
20 % 2 = 0  20 / 2 = 10
10 % 2 = 0  10 / 2 = 5
5  % 2 = 1  5  / 2 = 2
2  % 2 = 0  2  / 2 = 1
1  % 2 = 1  1  / 2 = 0
0  % 2 = 0  0  / 2 = 0

So 81 in binary is 01010001. Note that we extract exactly 8 bits because we assume that our character takes up 8 bits of memory.

Now, returning to our picture, we first zero out the least-significant bits (i.e., the rightmost ones) from each pixel by dividing and multiplying each pixel value by 2.

#27bc30 #ffae00 #27bc30 #ffae00 #ffae00 #27bc30 #ffae00 #27bc30

Then we add the bits of our character into each pixel, one bit per pixel. To make our implementation easier, we proceed by adding the least-significant bit of the character value (right-most) to the first pixel and then adding successive bits right-to-left to pixel values left-to-right.

#27bc30 + 1 = #27bc31 #ffae00 + 0 = #ffae00 #27bc30 + 0 = #27bc30 #ffae00 + 0 = #ffae00 #ffae00 + 1 = #ffae01 #27bc30 + 0 = #27bc30 #ffae00 + 1 = #ffae01 #27bc30 + 0 = #27bc30

The final values of these pixels is our encoded image which we would then write to disk.

The decoding process is similar. Suppose we have the encoded image:

#27bc31 #ffae00 #27bc30 #ffae00 #ffae01 #27bc30 #ffae01 #27bc30

We create an array of 8 elements to store 8 bits, then read off the least-significant bit from each pixel and store that bit in the array.

[1][0][0][0][1][0][1][0]

Recall that since we stored the bits in reverse order with the least-significant bit in front, we recover the bits in the same order. In this form, we can easily compute the integer value that these series of bits represents.

   [1]     [0]     [0]     [0]     [1]     [0]     [1]     [0]
  1*2^0 + 0*2^1 + 0*2^2 + 0*2^3 + 1*2^4 + 0*2^5 + 1*2^6 + 0*2^7
= 1     + 0     + 0     + 0     + 16    + 0     + 64    + 0
= 81

Now that we have the character value encoded in the pixels of the image, we can use asciitable.com to convert the character value 81 to the character 'Q' which is what we started with!

Part 1: the ImageDecoder Program

First, write a program ImageDecoder that prompts the user for a PNG image file and a (text) output file, decodes the characters hidde in the lower-order bits of each pixel of the image file and prints those characters to the output file.

Let's summarize the decoding process:

  1. Get the image pixel data as an array of integers.
  2. While there are still pixels left to read:
    • Read the next 8 pixels from the array.
    • Extract the 8 bits from those 8 pixels.
    • Convert those 8 bits into a character and print that character to the output file

For file output, use the PrintStream class as dicussed in lecture and in the book.

In Java, we represent images as instances of the BufferedImage class found in the java.awt.image package. To create instances of such objects from files, we must use the ImageIO class of the javax.imageio package. ImageIO has a public static method read that takes a File object and loads that image file as a BufferedImage. Thus, to load an image, say sample.png into your program, you would write:

BufferedImage img = ImageIO.read(new File("sample.png"))
ImageIO.read throws a new checked exception, an IOException that is found in the java.io package. So in the method that contains this line, you will need to add a throws IOException clause to the method signature similarly with Scanners and FileNotFoundException.

Once you have a BufferedImage object, you can pull out the pixel data by using the getRGB method:

// Returns an int[] containing a copy of the pixel data of the BufferedImage
getRGB(startX, startY, w, h, rgbArray, offset, scansize);

The different arguments of getRGB allow you to pull out only parts of the image data rather than the whole thing. However, we actually want the whole thing, so you should provide the following arguments to getRGB:

startX   = 0
startY   = 0
w        = the width of the BufferedImage
h        = the height of the BufferedImage
rgbArray = null
offset   = 0
scansize = the width of the BufferedImage

To get the width and height of the BufferedImage you can call the getWidth() and getHeight() methods of that BufferedImage.

Format and Testing

Here is a sample run of the decoder decoding the the first sample file:

This program reads in a png image file
and extracts a hidden message from it encoded
in the image's lower-order bits.  It then
writes that message to a file.

Enter image filename: sample1-enc.png
Enter output filename: out.txt
Hidden message written to out.txt!

The console output of the program is relatively uninteresting, but you should reproduce the format of this output exactly anyways.

Like previous homeworks, your prompting should be generous. In this case, the image file that the user enters should exist and the output file either does not exist or if does exist, it is writable. The exists(), canRead(), and canWrite() methods of the File class should be useful for checking these properties of the user input. Note that you do not need to check file extensions (which we learned in class are not important) nor check to see, e.g., if the image the user provide is actually an image file versus text. If they do provide an "image" file that Java can't load, then an IOException will be thrown.

The format of the generous prompting should be as follows, e.g., for the image filename:

Enter image filename: cheese
Invalid filename.  Enter image filename: example1-enc.png
Enter output filename: lockedfile.png
Invalid filename.  Enter output filename:

To test your decoder program, we've provided five sample images and texts (sampleX.txt and sampleX-enc.png respectively) in addition with the originals (sampleX.png) contained in the following zip file:

sample1-enc.png is similar to the image example we used to explain the encoding/decoding process in the previous section. The only difference is that the color values of the image are slightly different than in the example above (mostly to make the math easier). sample2-enc.png is another small image (45x1) that encodes a single word. The remaining examples are actual images containing texts of various sizes.


Can you find the message hidden inside? (sample3-enc.png)

Design

At a minimum, your ImageDecoder class should contain the following methods:

Beyond these two methods, you should have one additional, useful helper method in your program.

Part 2: the ImageEncoder Program

Now that we've written an image decoder, let's write the dual program, ImageEncoder that takes a text file, png image file, and output image file, and encodes the text in the image, saving the resulting image to the output file.

The encoding process can be summarized as follows:

  1. Get the image pixel data as an array of integers.
  2. For simplicity's sake, linearize the text file by storing it as a single String, preserving the exact structure of the file including whitespace.
  3. While there are still pixels left to read:
    • Read the next character from the text and translate it into its 8-bit representation.
    • Read the next 8 pixels from the array, clear them, and insert the character's bits into these pixels. Note that as per our convention, the lowest-order bit appears in the first of these 8 pixels. If there was no character to insert (because we are done processing the text), then we still clear the lower bits of the pixels anyways without inserting any data
  4. Afterwards, write the new image data to the output file.

In part 1, we used the BufferedImage and ImageIO classes to load and read the pixel data from an image. Here we will do the same thing, but we also need functionality to write image data.

The one additional detail we need to worry about when encoding text is normalizing the color model of our image. As we alluded to previously, pixel data can be encoded in a variety of ways. It turns out that some of these models (in particular, indexed color models that do not store color values directly for each pixel but instead offsets into an array that themselves contain the color values) are not amendable to the approach of bit stealing. To avoid these problems, we'll normalize the BufferedImage we create from the image file to be in the RBG color format that we want before we start processing it. (Note that we assume that an encoded image file is in the correct format already, so we don't need to do this for the decoding process).

To normalize the BufferedImage, we must do the following:

  1. Create a new empty BufferedImage with the correct color model. If our original BufferedImage is stored in img, the incantation to do this is:
    BufferedImage norm = new BufferedImage(img.getWidth(), img.getHeight(),
                                           BufferedImage.TYPE_INT_ARGB);
    
    where BufferedImage.TYPE_INT_ARGB is a static constant of the BufferedImage class that is used to say "I want a RGB color model for this image".
  2. Copy the old image into this new, normalized image. It turns out that BufferedImages have Graphics objects associated with them similarly to the DrawingPanels we've used earlier in the class. We can simply grab that Graphics object and use the drawImage method to draw img onto norm:
    norm.getGraphics().drawImage(img, 0, 0, null);
    
    the last argument of drawImage is an optional ImageObserver object that we don't need. So we pass in the null value in its place.

Format and Testing

The data archive samples.zip linked above also includes the original images, sampleX.png used to generate each of the encoded images, sampleX-enc.png that you used in the previous program. You should try to encode the related sampleX.txt files using ImageEncoder then check to see if everything worked by decoding the output with ImageDecoder and comparing the result with the original text file. You should also feel free to try this on other png image files and texts that you design yourself, find on the Internet, etc. to test your programs further.

Here is a sample run of the encoder encoding the first sample file:

This program reads in a text file and a png image
file and outputs a new image that is a copy of the
image as a png except with the text encoded in the
image's lower-order bits.

Enter text file: sample1.txt
Enter input image file: sample1.png
Enter output image file: sample1-enc.png

Written sample1-enc.png successfully!

Like the decoder, the console output is uninteresting, but reproduce it exactly anyways.

Similarly with ImageDecoder, your prompting should be generous. The input files should exist and be readable and the output file should either not exist or exist but be writable. You do not need to check either the file extensions or the actual format of the files in question. The format of the generous prompting is identical to that of ImageDecoder.

Design

At a minimum, your ImageEncoder class should contain the following methods:

Beyond these two methods, you should have two additional, useful helper methods in your program.

Extra Credit: Solve the Mystery

Update: after days of trying to recover the data from the hard drive, all the FBI was able to recover was this single image of a ceremonial goat.

It's a long shot, but you should take your trusty ImageDecoder and see if you can uncover any clues from the image. If you're able to follow any trails that Peter-Michael left and find the identities of the Mango Haters of America, the FBI will pull some strings and get you one extra credit point.

The FBI investigation ends on Wednesday 11/23, 11:59:59 am, just in time for Thanksgiving break. We need to solve the mystery before then!

(Practically speaking, this means that the due date for homework is still this Friday 11/18. The due date for the extra credit is 11/23.)

Late Days + Getting Started

Late days: this homework 7 inconveniently lines up around the time of the second midterm. Also, this is the penultimate homework, so you may want to use your late days now while you still have homeworks to burn. Because of these two facts, we're making an exception for late days used on this homework. Consider 11-19 through 11-21 as "free from homework" days. That is, I expect everyone to be studying for the exam on these two days, so I won't count these days for the purposes of homework submission.

In particular, if you decide to use one late day on this assignment (or elect to turn in the homework late without a late day with the -2 point penalty) then the homework will be due on 11-22, the day after the midterm. Using an additional late day will push the deadline out to 11-23 the final day of classes before Thanksgiving break.

Getting started: The previous homework was long but managable if you employed good decomposition skills. This homework is comparatively short in terms of code written, but the code itself is more complex. Here, decomposition helps you test difficult pieces of code so that you can isolate eventually bugs that you find in your program.

For example, suppose your write the second suggested method for ImageEncoder that translate a char into a bit array, call it charToBits(ch). Once that is complete, you should test charToBits(ch) in isolation from the rest of your program to ensure that it works. To do this, you should temporarily comment out the contents of your main method and write in calls to charToBits with various arguments and inspect the return values to see if they match what you want.

Otherwise, you should proceed as normal through this assignment:

  1. Understand what you need to do in this assignment by doing at least one example by hand. Again, it is difficult to tell a computer to do something that you haven't done yourself yet.
  2. In addition to the helper methods we tell you to implement, think of additional helper methods that you'll need and write and test those helper methods in isolation as described above.
  3. Once you have enough helper methods to make progress, try to put them together in main. From there repeat and refine until your program is complete.

Additional imports: We introduce a number of new classes in this homework for the purposes of image processing. Here is a summary of those classes and methods that you will need to use:

If you use these classes, make sure to include the appropriate import statements at the top of your Java file.

Submission

Please submit your Java source files, ImageDecoder.java and ImageEncoder.java in addition to any extra files you create for the extra credit electronically via the course website.

You will need to place these files into a zip archive called hw07.zip and submit that archive. On Windows you can create a new "Compressed (zipped) Folder" by right-clicking your desktop and selecting "New". On OSX, you can select your files in Finder, right-click and select "Compress". If you decide to only submit LootGenerator.java, you will still need to put it into a zip archive and submit that archive instead.