CloudDreamR - 009

I am continuing to work with camera data and I have been finding ways of using that data with ImageMagick.

Reading further into the documentation I am just beginning to realise how immensely powerful ImageMagick, especially as an opensource tool.  I've been experimenting with the "verbose" command which prints out file data from the RAW images I have been capturing.

Here is an example of some of the detailed data I can access from my RAW files

30042116.jpg
Image: 20170930042116.jpg
  Format: JPEG (Joint Photographic Experts Group JFIF format)
  Mime type: image/jpeg
  Class: DirectClass
  Geometry: 3280x2464+0+0
  Resolution: 72x72
  Print size: 45.5556x34.2222
  Units: PixelsPerInch
  Type: TrueColor
  Endianess: Undefined
  Colorspace: sRGB
  Depth: 8-bit
  Channel depth:
    red: 8-bit
    green: 8-bit
    blue: 8-bit
  Channel statistics:
    Pixels: 8081920
    Red:
      min: 37 (0.145098)
      max: 243 (0.952941)
      mean: 109.887 (0.430928)
      standard deviation: 37.8984 (0.148621)
      kurtosis: 2.09037
      skewness: 1.56838
    Green:
      min: 32 (0.12549)
      max: 237 (0.929412)
      mean: 99.3588 (0.389642)
      standard deviation: 36.1291 (0.141683)
      kurtosis: 3.51254
      skewness: 1.88898
    Blue:
      min: 36 (0.141176)
      max: 243 (0.952941)
      mean: 106.87 (0.419098)
      standard deviation: 36.5096 (0.143175)
      kurtosis: 2.6352
      skewness: 1.69613
  Image statistics:
    Overall:
      min: 32 (0.12549)
      max: 243 (0.952941)
      mean: 105.372 (0.413223)
      standard deviation: 36.8536 (0.144524)
      kurtosis: 2.7792
      skewness: 1.72331
  Rendering intent: Perceptual
  Gamma: 0.454545
  Chromaticity:
    red primary: (0.64,0.33)
    green primary: (0.3,0.6)
    blue primary: (0.15,0.06)
    white point: (0.3127,0.329)
  Background color: white
  Border color: srgb(223,223,223)
  Matte color: grey74
  Transparent color: black
  Interlace: None
  Intensity: Undefined
  Compose: Over
  Page geometry: 3280x2464+0+0
  Dispose: Undefined
  Iterations: 0
  Compression: JPEG
  Quality: 100
  Orientation: Undefined
  Properties:
    date:create: 2017-09-30T19:19:08+13:00
    date:modify: 2017-09-30T17:21:22+13:00
    exif:ApertureValue: 20000/10000
    exif:BrightnessValue: 249/100
    exif:ColorSpace: 1
    exif:ComponentsConfiguration: 1, 2, 3, 0
    exif:DateTime: 2017:09:30 04:21:21
    exif:DateTimeDigitized: 2017:09:30 04:21:21
    exif:DateTimeOriginal: 2017:09:30 04:21:21
    exif:ExifImageLength: 2464
    exif:ExifImageWidth: 3280
    exif:ExifOffset: 192
    exif:ExifVersion: 48, 50, 50, 48
    exif:ExposureMode: 0
    exif:ExposureProgram: 3
    exif:ExposureTime: 78329/1000000
    exif:Flash: 0
    exif:FlashPixVersion: 48, 49, 48, 48
    exif:FNumber: 20000/10000
    exif:FocalLength: 30390/10000
    exif:ImageLength: 2464
    exif:ImageWidth: 3280
    exif:InteroperabilityOffset: 906
    exif:ISOSpeedRatings: 125
    exif:Make: RaspberryPi
    exif:MakerNote: 101, 118, 61, 45, 49, 32, 109, 108, 117, 120, 61, 45, 49, 32, 101, 120, 112, 61, 55, 56, 51, 50, 57, 32, 97, 103, 61, 54, 51, 55, 32, 102, 111, 99, 117, 115, 61, 50, 53, 53, 32, 103, 97, 105, 110, 95, 114, 61, 49, 46, 50, 54, 57, 32, 103, 97, 105, 110, 95, 98, 61, 49, 46, 52, 49, 52, 32, 103, 114, 101, 101, 110, 110, 101, 115, 115, 61, 45, 56, 57, 32, 99, 99, 109, 61, 54, 48, 55, 56, 44, 45, 50, 48, 52, 52, 44, 54, 56, 44, 45, 49, 49, 57, 56, 44, 53, 52, 50, 56, 44, 45, 49, 51, 48, 44, 50, 55, 50, 44, 45, 51, 57, 49, 54, 44, 55, 55, 52, 52, 44, 48, 44, 48, 44, 48, 32, 109, 100, 61, 48, 32, 116, 103, 61, 50, 54, 49, 32, 50, 54, 49, 32, 111, 116, 104, 61, 48, 32, 48, 32, 98, 61, 48, 32, 102, 61, 50, 54, 49, 32, 50, 54, 49, 32, 102, 105, 61, 48, 32, 73, 83, 80, 32, 66, 117, 105, 108, 100, 32, 68, 97, 116, 101, 58, 32, 65, 117, 103, 32, 32, 56, 32, 50, 48, 49, 55, 44, 32, 49, 50, 58, 48, 52, 58, 52, 54, 32, 86, 67, 95, 66, 85, 73, 76, 68, 95, 73, 68, 95, 86, 69, 82, 83, 73, 79, 78, 58, 32, 49, 57, 54, 48, 54, 51, 52, 51, 56, 100, 55, 98, 98, 48, 55, 54, 57, 56, 54, 49, 99, 56, 54, 102, 57, 52, 56, 97, 101, 54, 53, 51, 55, 50, 55, 97, 100, 98, 55, 101, 32, 40, 99, 108, 101, 97, 110, 41, 32, 86, 67, 95, 66, 85, 73, 76, 68, 95, 73, 68, 95, 85, 83, 69, 82, 58, 32, 100, 99, 52, 32, 86, 67, 95, 66, 85, 73, 76, 68, 95, 73, 68, 95, 66, 82, 65, 78, 67, 72, 58, 32, 109, 97, 115, 116, 101, 114, 32
    exif:MaxApertureValue: 20000/10000
    exif:MeteringMode: 2
    exif:Model: RP_imx219
    exif:ResolutionUnit: 2
    exif:ShutterSpeedValue: 3674309/1000000
    exif:thumbnail:Compression: 6
    exif:thumbnail:ImageLength: 48
    exif:thumbnail:ImageWidth: 64
    exif:thumbnail:InteroperabilityIndex: R98
    exif:thumbnail:JPEGInterchangeFormat: 1042
    exif:thumbnail:JPEGInterchangeFormatLength: 24576
    exif:thumbnail:ResolutionUnit: 2
    exif:thumbnail:XResolution: 72/1
    exif:thumbnail:YResolution: 72/1
    exif:WhiteBalance: 0
    exif:XResolution: 72/1
    exif:YCbCrPositioning: 1
    exif:YResolution: 72/1
    jpeg:colorspace: 2
    jpeg:sampling-factor: 2x2,1x1,1x1
    signature: b600a2a2dbd22e0545ed6127baccaaaaa68cbdfcbb6bcd983c8def1b6388ca03
  Profiles:
    Profile-exif: 25624 bytes
  Artifacts:
    filename: 20170930042116.jpg
    verbose: true
  Tainted: False
  Filesize: 14.55MB
  Number pixels: 8.082M
  Pixels per second: 42.54MB
  User time: 0.190u
  Elapsed time: 0:01.190

As I think about other ways of visualising the camera data I have been digging deeper into the ImageMagick documentation.  I've been learning about different colorspaces and how they can be represented in different ways.  

As a filmmaker I love the concept of animating frames.  Essentially that what filmmaking is - sequencing still images to create movement.  Now that I've seen some of ideas about graphing data come to fruition I am starting to think about how I translate them into animations, or moving images.  So I dug a little deeper into the ImageMagick documentation and learnt about different colorspaces and how they can be displayed in different ways.  You know where these are heading...

 

colorwheel_HSL.png
colorwheel_LCHab.png

CloudDreamR - 008

I have found working with opensource software an amazing experience on many levels.  There is now nagging feeling that I'll need to paying my monthly subscription soon, and even less of a nagging feeling that I haven't used it enough to feel as though I've got my my money's worth.

Most of the programs I am using now operate from Terminal, or via the command line.  I's been a huge step to commit to working in this way and there have been times I've ended up feeling stranded, like I was drowning and that I should give.  But I've pushed on enough to know that I can't turn back now.

Two programs that I have been for the image work are DCraw and ImageMagick.  I use these to access the RAW data available in the images I capture on the Raspberry Pi camera.  About six months I spent an evening online with the friend of mine as they worked their way through the options (in scripting they're referred to as flags) available in terms of file output.

I taken photographs for over 30 years now, and have been using digital since its inception.  So to access RAW data in such a pure form was a revelation.  With one or two lines of code I could access image file before the camera or computer even interpolated the RGB channels into a color image.  It was amazing.

For example here is a "verbose" print out of the some of the data I can extract from a file via ImageMagick.

30042116.jpg
Image: 20170930042116.jpg
  Format: JPEG (Joint Photographic Experts Group JFIF format)
  Mime type: image/jpeg
  Class: DirectClass
  Geometry: 3280x2464+0+0
  Resolution: 72x72
  Print size: 45.5556x34.2222
  Units: PixelsPerInch
  Type: TrueColor
  Endianess: Undefined
  Colorspace: sRGB
  Depth: 8-bit
  Channel depth:
    red: 8-bit
    green: 8-bit
    blue: 8-bit
  Channel statistics:
    Pixels: 8081920
    Red:
      min: 37 (0.145098)
      max: 243 (0.952941)
      mean: 109.887 (0.430928)
      standard deviation: 37.8984 (0.148621)
      kurtosis: 2.09037
      skewness: 1.56838
    Green:
      min: 32 (0.12549)
      max: 237 (0.929412)
      mean: 99.3588 (0.389642)
      standard deviation: 36.1291 (0.141683)
      kurtosis: 3.51254
      skewness: 1.88898
    Blue:
      min: 36 (0.141176)
      max: 243 (0.952941)
      mean: 106.87 (0.419098)
      standard deviation: 36.5096 (0.143175)
      kurtosis: 2.6352
      skewness: 1.69613
  Image statistics:
    Overall:
      min: 32 (0.12549)
      max: 243 (0.952941)
      mean: 105.372 (0.413223)
      standard deviation: 36.8536 (0.144524)
      kurtosis: 2.7792
      skewness: 1.72331
  Rendering intent: Perceptual
  Gamma: 0.454545
  Chromaticity:
    red primary: (0.64,0.33)
    green primary: (0.3,0.6)
    blue primary: (0.15,0.06)
    white point: (0.3127,0.329)
  Background color: white
  Border color: srgb(223,223,223)
  Matte color: grey74
  Transparent color: black
  Interlace: None
  Intensity: Undefined
  Compose: Over
  Page geometry: 3280x2464+0+0
  Dispose: Undefined
  Iterations: 0
  Compression: JPEG
  Quality: 100
  Orientation: Undefined
  Properties:
    date:create: 2017-09-30T19:19:08+13:00
    date:modify: 2017-09-30T17:21:22+13:00
    exif:ApertureValue: 20000/10000
    exif:BrightnessValue: 249/100
    exif:ColorSpace: 1
    exif:ComponentsConfiguration: 1, 2, 3, 0
    exif:DateTime: 2017:09:30 04:21:21
    exif:DateTimeDigitized: 2017:09:30 04:21:21
    exif:DateTimeOriginal: 2017:09:30 04:21:21
    exif:ExifImageLength: 2464
    exif:ExifImageWidth: 3280
    exif:ExifOffset: 192
    exif:ExifVersion: 48, 50, 50, 48
    exif:ExposureMode: 0
    exif:ExposureProgram: 3
    exif:ExposureTime: 78329/1000000
    exif:Flash: 0
    exif:FlashPixVersion: 48, 49, 48, 48
    exif:FNumber: 20000/10000
    exif:FocalLength: 30390/10000
    exif:ImageLength: 2464
    exif:ImageWidth: 3280
    exif:InteroperabilityOffset: 906
    exif:ISOSpeedRatings: 125
    exif:Make: RaspberryPi
    exif:MakerNote: 101, 118, 61, 45, 49, 32, 109, 108, 117, 120, 61, 45, 49, 32, 101, 120, 112, 61, 55, 56, 51, 50, 57, 32, 97, 103, 61, 54, 51, 55, 32, 102, 111, 99, 117, 115, 61, 50, 53, 53, 32, 103, 97, 105, 110, 95, 114, 61, 49, 46, 50, 54, 57, 32, 103, 97, 105, 110, 95, 98, 61, 49, 46, 52, 49, 52, 32, 103, 114, 101, 101, 110, 110, 101, 115, 115, 61, 45, 56, 57, 32, 99, 99, 109, 61, 54, 48, 55, 56, 44, 45, 50, 48, 52, 52, 44, 54, 56, 44, 45, 49, 49, 57, 56, 44, 53, 52, 50, 56, 44, 45, 49, 51, 48, 44, 50, 55, 50, 44, 45, 51, 57, 49, 54, 44, 55, 55, 52, 52, 44, 48, 44, 48, 44, 48, 32, 109, 100, 61, 48, 32, 116, 103, 61, 50, 54, 49, 32, 50, 54, 49, 32, 111, 116, 104, 61, 48, 32, 48, 32, 98, 61, 48, 32, 102, 61, 50, 54, 49, 32, 50, 54, 49, 32, 102, 105, 61, 48, 32, 73, 83, 80, 32, 66, 117, 105, 108, 100, 32, 68, 97, 116, 101, 58, 32, 65, 117, 103, 32, 32, 56, 32, 50, 48, 49, 55, 44, 32, 49, 50, 58, 48, 52, 58, 52, 54, 32, 86, 67, 95, 66, 85, 73, 76, 68, 95, 73, 68, 95, 86, 69, 82, 83, 73, 79, 78, 58, 32, 49, 57, 54, 48, 54, 51, 52, 51, 56, 100, 55, 98, 98, 48, 55, 54, 57, 56, 54, 49, 99, 56, 54, 102, 57, 52, 56, 97, 101, 54, 53, 51, 55, 50, 55, 97, 100, 98, 55, 101, 32, 40, 99, 108, 101, 97, 110, 41, 32, 86, 67, 95, 66, 85, 73, 76, 68, 95, 73, 68, 95, 85, 83, 69, 82, 58, 32, 100, 99, 52, 32, 86, 67, 95, 66, 85, 73, 76, 68, 95, 73, 68, 95, 66, 82, 65, 78, 67, 72, 58, 32, 109, 97, 115, 116, 101, 114, 32
    exif:MaxApertureValue: 20000/10000
    exif:MeteringMode: 2
    exif:Model: RP_imx219
    exif:ResolutionUnit: 2
    exif:ShutterSpeedValue: 3674309/1000000
    exif:thumbnail:Compression: 6
    exif:thumbnail:ImageLength: 48
    exif:thumbnail:ImageWidth: 64
    exif:thumbnail:InteroperabilityIndex: R98
    exif:thumbnail:JPEGInterchangeFormat: 1042
    exif:thumbnail:JPEGInterchangeFormatLength: 24576
    exif:thumbnail:ResolutionUnit: 2
    exif:thumbnail:XResolution: 72/1
    exif:thumbnail:YResolution: 72/1
    exif:WhiteBalance: 0
    exif:XResolution: 72/1
    exif:YCbCrPositioning: 1
    exif:YResolution: 72/1
    jpeg:colorspace: 2
    jpeg:sampling-factor: 2x2,1x1,1x1
    signature: b600a2a2dbd22e0545ed6127baccaaaaa68cbdfcbb6bcd983c8def1b6388ca03
  Profiles:
    Profile-exif: 25624 bytes
  Artifacts:
    filename: 20170930042116.jpg
    verbose: true
  Tainted: False
  Filesize: 14.55MB
  Number pixels: 8.082M
  Pixels per second: 42.54MB
  User time: 0.190u
  Elapsed time: 0:01.190

Here's an example of a JPG versus a RAW image shot on the Raspberry PiCamera.

JPEG file

JPEG file

RAW file

RAW file

CloudDreamR - 007

The CloudDreamR project aims to interact with Data.  Most of the initial part of the design process has been around assembling and activating the hardware, and as that comes together I am finding myself looking forwards to the time when I begin to access the data flow.

With this in mind I found myself with a gap in the hardware implementation and decided to look into the software workflow, what that might look and how it might work.

Anticipating the types of data I might be able to work with from the Raspberry Pi camera as wellthe Environmental sensors I have on order, I began some experiments with gnuplot.

I like reading the history around computer hardware and software, so I was drawn to gnuplot because is one of those 'early' opensource programs.   It's design began out of necessity, and was initiated by two classmates in the late 1980's.

I installed it on Ubuntu 16.04 and began reading the documentation and following some examples in Terminal.

By the end of the day I had begun to achieve some of the ideas I had been conceptualizing earlier.  Could I make a graph with out keys, or codes, on numbers?  Like just and image?

It took a while but I began to get some simple results that I liked.  And I figured out that I could manipulate the colour in ways that I want to going forward.

 

 

 

experiment 64.png
experiment 42.png
experiment2.png
experiment 8.png

CloudDreamR - 006

Big steps forward.  Another Tuesday evening Fab Lab session and I managed to solder the pins (bady) onto the VEML6070 UV sensor.  Also borrowed a breadboard and some jacks.  

_DSC8240.JPG

 

Here are the pics of the setup via the breadboard.

 

Once I figured out how to get the jacks running between the sensor, breadboard and the SENS01 Raspberry Pi it was time to tackle the interface via Terminal.  Wow - was that daunting.  And I'm still struggling with a couple lines of code.  But the learning curve was amazing, and I've begun to get some vague understanding about how the syntax works in Python.  So much to learn!

 

_DSC8253.JPG
_DSC8255.JPG

CloudDreamR - 005

Today I realized that I'd left a crucial part out of the power setup.  The PoE Injector.  I ordered the Splitter but obviously I didn't understand the flow well enough.  After a quick confab with a colleague we identified the missing part.  

Then, would you believe a friend dropped over on Sunday, looked at the setup, runs down his car and brings back and Injector - unbelievable!  So now the system is complete - and running!

_DSC8227.JPG

CloudDreamR - 004

Today I was focusing on the attaching the Conduit adaptor so that I can run the HDMI and Ethernet cables into the enclosure.  I always get carried away with the design side of things and that tripped me up today. 

When I dropped round to a friends place to attach the cable mount I realised that the enclosure was too small.  My friend reminded me that getting everything in there and running readily should be the main focus. There were workarounds, but then I figured the best thing to do was to upgrade the enclosure, (while I still have the receipt and the plastic bag!).   Having room to move is a real benefit - especially at this early stage.

A quick bike ride later and Global PC delivered not only a enclosure swap but also the one small, obscure item I was lacking to get rig going over the weekend - the micro USB adaptor for the PoE (Power over Ethernet) box - nice!

 

enclosure.JPG
_DSC8219.JPG

CloudDreamR - 003

Yesterday and today have I've been working away on getting an enclosure sorted for the SENS01.

Yesterday's work involved making a template for the Fab Lab laser cutter.  We scanned the enclosure into Autocad and used it to create develop a.dxf file.  With the .dxf file I can cut an acrylic base plate to mount the Raspberry Pi3 on. In the meantime I think I'll cut one out of some stray MDF in the garage just to keep things moving.

Paper & Pencil
Scan 1
Scan 2

CloudDreamR - 002

Fun day today.  Down to Bunnings and Global PC to source an enclosure of the SENS01 as well as gland for the cabling to run power and HDMI.

Also ordered the comms units for both SENS01 and DATA01.

And found a great electronics supplier down in Timaru.  Check them out.  

https://nicegear.co.nz

 

Enclosure 01

CloudDreamR – One

My package from Adafruit touched down and was ready for pickup when I rode into the FabLab today on my bike.

I’m excited about getting the gear and a little apprehensive about figuring it out and making it go.

It’s not every day I decide to delve into building electronic computer devices – especially ones that are designed to transmit data to each other.

Here’s what arrived in the mail:

 

image2.jpg
image5.jpg
image4.jpg

And here’s what I ended up with today...2x Raspberry Pi3’s ready to rock.  For more details and pics from the unpack go to my blog at www.shbrodie.wixsite.com/clouddreamr

image1.jpg
image3.jpg

CloudDreamR

Creative Concept

A Raspberry Pi timelapse data system including Visible, Infrared and UV sensors to monitor cloud and weather patterns over long time periods.  RAW data streams will be presented in animated and sound forms.

Collaborators: Steve Brodie, and others to be decided.