Data Wrangler
One of the services I offer is that of Data Wrangler.
This means getting video files safely from the camera to the computer.
I worked on the Isle of Man TT Races for Greenlight Television in this capacity, the below post summarises my approach to this role and how it can help a production of any size.
The role crosses over into DIT (digital imaging technician) to some extent, I can help advise production teams (specifically camera operators and the Director and Producer) on work in progress. Film making is often a fast-paced environment to work in and having someone checking the rushes for consistency and quality can be of great benefit during a shoot. The key roles I fulfil in this capacity are:
Establishing and maintaining control of workflow: Larger productions have their own established workflow which I will fit into. Smaller productions may not and so I can help work with the production team to establish a workflow that best suits the needs of the project.
Problem-solving: Filming in the field often throws up unexpected challenges, be they creative or technical. Having a calm pair of hands is helpful to fix kit and cable connections and ensure cameras are set up correctly.
Digital cameras and computers: Having a broad understanding of file formats, computers, and media and the best way to get data onto and off of devices is crucial.
Communication: Working methodically within a high-stress environment, and having the confidence to speak up to ensure that the crew stick to the rule book, especially when they are under pressure themselves.
Attention to detail: Practice what you preach, ensure that media, files and folders are labelled accurately, that data is transferred and copied without loss, and that corruptions are spotted and addressed.
How can you guarantee that your data will be transferred and backed up correctly?
I run ShotPut Pro on a state-of-the-art Apple MacBook Pro laptop to ensure that data is transferred and backed up accurately. The simple user interface and robust copy speeds make it indispensable for today's file-based workflows. Using it I can make multiple, verified copies from one source. Guaranteeing that what has been recorded onto the camera’s card is transferred precisely.
Software: ShotPut Pro, the industry-standard software for offloading media files. Creating MHL Reports, Checksum Verified Copies or PDF Reports With Thumbnails and Metadata.
Hardware: MacBook Pro with M1 Max processor for the fastest data transfer times.
Establishing workflow:
Robust workflow is key to any large (or small) scale data wrangling assignment. At the outset of any project I establish the criteria under which the entire production team will work including:
Pre-Production:
Work with production to determine the most appropriate file types, codecs and bitrates to shoot in camera and ensure consistency.
Design a flowchart to plan how many machines, card readers, cables and connectors will be needed, including contingency for redundancy or kit failure.
Source correct card readers for media, ensuring super-fast cables and connectors where needed.
Calculate the required amount of hard drive space so that drives can be purchased at the best price ahead of time.
Format drives according to the intended workflow (Mac or PC etc).
Production:
Establishing a protocol for crews and camera card roll numbers.
Taping card slots.
Recording timecode based shot lists for production to keep track of whats been covered and for the edit team to refer to later.
Transferring Media (physical):
Cards to be secured in cases and labelled the moment they leave camera.
Card cases placed in sealed envelopes which are marked with appropriate information (roll number, date, crew etc).
Shot lists to accompany cards in digital form.
Transferring Media (digital):
Envelopes placed in RED in-tray for processing.
Cards are removed and processed one at a time. Tape from card or case stuck to the card reader to identify which card is being processed.
ShotPut Pro used to transfer media to a preformatted hard drive.
Await confirmation from ShotPut Pro that the copy has been successful and logs have been generated.
Move the card onto the backup phase and repeat.
Once card is backed-up record this as completed in a digital log and file the envelope in a folder (in case it needs to be checked at later date).
Visual check that the hard drives match in terms of file sizes.
Transfer copied card into GREEN out-tray (located on a separate table) for camera crew to collect and shoot on again.
Post-Production
Create proxy files if required (these can also be shot in camera).
Review daily footage for technical consistency across camera teams (frame rates, resolution, audio, video).
Begin building assembly edits for the Director to review creatively (style, lens choice, storylines etc).
Put together sizzle reels to inspire the crew (helpful on longer projects).
Note RED in-tray behind Mac with an envelope ready for processing. This project required around 40 TB of data to be processed. It is important to stay well organised. Physical space is often limited in production offices.
Note GREEN out-tray for cards that have been copied and are ready to shoot on again. Green tape is helpful to mark media, cards, handy-cams, GoPro’s, drones etc. that have been processed and are ready to go back out into the field. A folder is used to store completed envelopes.