A useful and efficient way to manage data and photo files
Many photographers face a lot of data and images after a while. In such a situation, proper data management is necessary with proper storage strategies.
Marcel Fauston, a professional nature photographer from the Netherlands, and the tips of this article are based on his personal experiences. He and his wife, Daniela, set up a Squiver company that organizes nature photography tours for small international groups at all levels and destinations.
This year, it was dubbed the photojournalist of the year, and in the past had won the Year of Nature Travel Awards.
When he started photography, he did not have a specific way to manage his mass data. He immediately entered the laptop immediately after taking pictures, and after copying, he copied all of them to the computer. Sometimes the data was backed up on the external hard disk.
After all, the hard drive of his computer was filled; so all the data was transferred to another external hard drive. Perhaps the capacity of the external HDDs seems to be high, but the memory was too early to fill. So, Austen provided two additional hard drives for images and two other hard drives for backup.
After some years, his data management system was full of hard drives in a variety of capacities and connectivity. In addition, his photos were not quite regular enough; consequently, his mind became more turbulent. Austen says:
I saw many photographers with similar problems. Some of them were unaware of this destiny, and those who were aware did not know the right solution. Right now, there are many questions about data management, how to download, organize, and back up images to me.
Austen in this article explains the implications of the trend: the method of data management from the stage of photography to the final backup.
Of course, this method is not necessarily the main general basis for backup. There are several ways to manage data, but this article only points to Austen’s priorities and experiences, and there may be better ways to do this.
On the road and the location of photography
On all trips, Austen takes a MacBook Pro and a minimum of three laptops. Two of the hard disks are drowning and include all the travel images.
In terms of data storage, disk drives are referred to disk logical hard drives in real-time physical hard drives.
The third drive contains all the processing files required on the road. Austen performs the following steps after photography:
1 Connect the HDD 1 to the MacBook: Create a new folder named the country you visited; Create another folder called Transport in the same folder.
2 Importing all the files (regardless of the type) into the Transport folder: The next step is to change the names of the images in the Lightroom software, but before that it is necessary to make this folder because the process of renaming directly from the hard drive in Lytmore is very timely.
3 Open Lightroom and click the Import and Transport button: Move the images to a new location by clicking Move, and then selecting the destination in the upper right corner of the window. In this example, the parent folder is named after the country where the photos are registered.
4 Enable the Rename Files option and use custom settings. In this example, the file is named as: the first two or three letters of the country’s name, the first two or three letters of the place of the photographer,
the date (which begins with the year), the consecutive number. This way you can see the time and location of the photo, without having to open the file; for example:
CHI_HUA_190315_666.NEF (image recorded in Huangshan China, March 15, 2019).
5 Add public keywords that apply to all input images. Specific keywords will be added in the next steps.
6 After the files are re-entered, the names of all the files are changed quickly, they enter into the destination folder and are organized according to the date.
In the end, you will see the Leitrim Austen library. In this library, he has a folder for each continent and is a folder of countries in the continental folders.
There are several folders in the folder in each country (depending on how many times you visited the country).
After downloading all the images, renaming and organizing them, you can connect HDD 2 to back it up.
Backup is a very necessary and desirable step because it may lose one of your hard disks, or other people will steal it, or break it.
There are several ways to back up information, Austen uses the Carbon Copy Cloner software to create an accurate copy of data.
In this way, in the event of a version failure, there is a completely identical version of it.
The Carbon Copy Cloner user interface is shown in the image below. Of course, merely plugging files (especially large file bits) can lead to their failure.
For example, Drag sometimes causes a large horizontal bar to appear in the middle of the photo, but with CCC software you can easily transfer files.
A few years ago, a camera member of Austen’s photography tour was stolen right on the last day of the tour. His contents included camera equipment, laptops, and even backup drives.
He lost everything, but this can be a good lesson: always keep your backup drives at a separate location.
For example, Austen holds one of the backup drives in a laptop computer and the other in a camera bag.
Depending on the location, it also carries one of the drives and places the other in the hotel room. As a result, even with theft of the hotel, it has one drive.
There are several options for choosing a hard drive. Austen usually uses LaCie Rugged hard disks that have high firmness.
He often takes photographs in conditions such as sand, humidity, high temperatures, rocks and rocks, which are not very suitable for electronic devices, but these drives are robust enough, but still care should be taken.
A golden monkey is followed by the Austen Hard Disk Drive, but it’s not a worry about the hard disk.
Austen during the trip also picks up and picks up at Lightroom, but postpones the final processing to the EIZO calibration monitor and home screen.
For example, it always arrives at Lytrum on the last day of your journey and draws a catalog from the travel directory, then stores the catalog on one of the external drives.
Upon arriving at home, the catalog enters the main library of Lightroom, and stores all the signs, stars, and settings made during the trip.
More megapixels, more data
Every day, the number of megapixels for new cameras increases. An increase in megapixels for large prints or photo cuts is a good thing, but as the megapixel increases, the amount of data increases.
For example, a lacing tiff or PSD image from the Nikon D850 can easily occupy 4GB.
Panaroma pictures will get worse. Now suppose you have hundreds of such images, you will need more memory with a capacity in these circumstances.
A few years ago, he wrote an article about LaCie 5big Thunderbolt 2 RAID storage.
The first paragraph of this article was about the number of recordings he recorded with different cameras in the one year olds.
In 2018, Austen recorded 72, 028 images with these cameras, with a total volume of 3.9 terabytes.
|Camera Model||Number of Pictures|
His figure in 2016 was 76, 328 with a total volume of 2 terabytes, but two years later, with the increase in megapixels of cameras, despite the diminution of 4,000 photos from this series, the volume of data almost doubled.
Now, he uses the 48-megapixel Nikon Z7, and the size of the image is expected to increase in 2019.
Technology is fast-paced. File sizes are increasing day by day. Regardless of the type of memory you intend to purchase, make sure the next memory capacity is doubled.
If photography is not your main job and you only occasionally shoot for entertainment, it’s enough to have a multi-hard disk and a hard drive to back it up; but if your job is photography, especially wildlife photography, a bird or sports photographer, you will need another memory.
Austen had 25 TB of data stored when writing the article, but the question is how can I manage this volume of data?
Austen used years ago for the RAID system and chose LaBrite to launch LaCie. Then it upgrades to 6big and now it uses 6big Thunderbolt 3.
What is 6big and how does it work?
LaCie 6big Thunderbolt 3 is a kind of DAS memory or direct connection memory. This memory is a kind of external hard drive that consists of 6 drives and connects directly to the computer.
This memory is available in several different capacities of 12, 24, 36, 48, 60 and 84 terabytes, but its appearance is not like ordinary external hard drives. The 6big TB3 is a smart memory device with a variety of features.
In this system, a RAID controller provides multiple settings and modes as requested by the user. With this setting you can get the best performance.
Which memory is suitable for whom?
This type of memory is special for people who need a fast, flexible, and secure way to store a huge amount of data.
Most photographers begin their work with a hard drive, after they fill it up another hard drive.
And they continue. This data storage system is called JBOD or a large number of disks.
Data storage is very confusing in this way, and there is no worse way to protect data.
But the question is that a lot of data disappears if one of the hard drives crashes. Associating with DAS or NAS storage systems (similar to the DAS network connection storage method, with the difference that it is connected to a computer network), it is recommended with RAID.
RAID stands for the dual array of independent discs.
Using RAID memory is essential for professional photographers
This protective technology is used in external hard drives. In general, the higher the RAID level, the better the data security will be guaranteed.
A RAID-equipped system uses two or more hard drives to protect data against errors, and higher levels of RAID require more independent discs.
In such a system, in the event of a failure of one of the hardware, especially the hard drives, other components will continue to work.
Most professional photographers often emphasize the safety of their equipment and camera, but they do not care about the safety of their images.
Due to the use of a robust RAID system, it is recommended to be insured for pictures.
RAID consists of several levels, each with a certain level of redundancy, each of these levels affecting the speed of the work.
In sum, zero redundancy means maximum speed and redundancy maximum, meaning the lowest speed.
Depending on what you do, you can select your desired RAID level. A review of the most common RAID levels is provided.
RAID 0 is used to improve performance. This level is also known as disk striping. In RAID 0 data is written on multiple disks.
This means that instead of a disk, multiple disks will take over the task of controlling the computer, and since several drives read and write data, the system performance will be improved.
To implement this level of RAID, at least two disks are required. The fault tolerance is fault tolerance, which means that a disk crash affects the whole array and increases the chance of data loss or corruption.
RAID 1 This level is known as the Disk Error Level (Mirroring Disk), a disk that is exactly the same as another disk mirror.
In RAID 1, the data is synchronized from the disk to another disk, and the terms are dumped from them. If a disk fails, another disk will continue to work.
This level is the simplest way to implement fault tolerance and has a relatively low cost.
The RAID 1 defect has a slight downgrade. At least two disks are required to implement RAID 1 hardware. On the other hand, RAID 1 will reduce the total disk capacity by half.
RAID 5 is the most commonly used RAID configuration for today. The performance of this RAID level is better than Mirroring and error tolerance.
In RAID 5, data and parity (referred to as hard drives used in the RAID array for error tolerance) are distributed between three or more disks.
If the disk fails or crashes, the data is reconstructed from the distributed data and the parity block is integrated and automated.
Even if the life of the disc is over, the system will be usable until the replacement of the drive is corrupted.
Another advantage of RAID 5 is its hot swappable feature. This means that in the event of a failure of one of the drives or arrays, the drive can be replaced without having to dump the DAS or NAS.
This feature is a good solution for error handling, because in the event of a drive failure, it can be rebuilt on new disks. RAID 5 performance is compatible with RAID 0 performance.
RAID 6 is identical to RAID 5, with the difference that due to the greater number of Parity blocks than the RAID 5, it is more stable, so if two disks fail, the system can still be used. At least four disks are required to build an RAID 6 array.
RAID 10 is also known as Nesting RAID. At this level of memory, the RAID 1 protection feature is combined with RAID 0 performance improvements.
This level offers the best performance, but it is very costly and requires twice as much disk space as needed. The minimum number of disks required to implement this level is 14 disks.
RAID 50, this RAID level is composed of RAID 0 racking with parity RAID 5. The result of the fast speed of RAID 0 streaming is a better RAID 50 performance than RAID 5, especially in the writing process.
On the other hand, its security is more than a RAID. If you need to tolerate the improved error, use high capacity and faster speeds than this RAID level. At least six hard drives are required for an RAID 50 array.
The RAID 50 array with a large number of hard drives has high storage capacity, and thus the time to initialize and reconstruct data increases.
RAID 5 automatically balances between memory space, performance, and data security, and is therefore the best-recommended storage option for all devices that have three or more internal storage.
The security and performance of this system is high and still maintains 80% of the disk space usability.
Even in the event of a disk failure, the hot swappable feature can be replaced without disconnection or without power outage.
Design and facilities
Maybe the appearance of memory is not important to you, but the 6big external memory is polished and matched to the Mackbook Pro model.
Asstone believes reliability in the choice of memory is a priority. For over two decades, he has been using LaCie HDDs.
Although these memories do not have a beautiful design, they have a high reliability instead.
These drives are horizontally aligned and accessible from the front.
External lines are not just part of the design, but they separate 6 drives. Drives can easily be removed from the front of the product, or moved.
Six rows of drive in the front of the memory. Hot swappable drives, easy access
Drives are installed on 6big and do not need to open them. You only have to open them when you want to replace older drives with larger drives.
The probability of failure of any of the hard drives is the same, and the main reason for choosing the 6big is the same. In RAID 5, if a drive fails, the data will not be lost, and in RAID 6 this will apply to the two drives, but most people expect the drives to never be corrupted.
While in reality the eternal life of the drives is not feasible, LaCie has the best available drives in the market, including Seagate IronWolf Pro. These drives are also called enterprise-level drives.
LaCie has a five year warranty for these drives along with data recovery services (Rescue Data Recovery Services).
In the event of a problem, the company will replace the drive and will retrieve lost data without additional charge.
One of the Seagate Ironwolf Pro’s Seventy-four Seven Thirds Drives in the 6big Austen Series
Due to the need for more memory capacity, Austen upgraded its memory system. On the other hand, by upgrading its storage system, it was able to take advantage of the latest advancements in computer technology, including Thunderbolt 3.
The Thunderbolt brand is a hardware intermediary developed by Intel (in collaboration with Apple) and allows external devices to be connected to a computer. Thunderbolt 3 is dramatically faster than Thunderbolt 2.
Larger files not only occupy more space, but their power consumption is also high. In the 6big system, there are six IronWolf Pro drives, each with a storage capacity of 14 terabytes, and therefore a total of 84 terabytes of storage capacity is available.
The Ashton file size is 25 terabytes, but in the 6big system, the volume is up to 36TB (6x6TB). In RAID 5, one disk is used for redundancy, and therefore there is only 30 terabytes for storage (5x6TB), so only 5 terabytes of free space remains to be filled in over two years.
On the other hand, RAID 5 is not only more common, it’s one of the safest and fastest file storage options.
Increasing the volume of Austen files, especially video files captured with the Nikon Z7, have a significant impact on memory.
Other RAID levels have high security and performance, so the next option can be a 84TB RAID 10 version (this level of RAID has the best performance, but its redundancy is greater than RAID 5).
Primary settings and performance
Austen uses a variety of technologies for his job, but the technical photography department does not have much interest in him. Creativity is a priority for him.
Cameras, computers, and other devices to achieve better results. He says:
I’m not interested in spending money on a RAID system or its settings, but this is necessary to achieve better results. The design and operation of the 6big system follows the slogan of simplicity.
Setting up 6big is not an easy task. You must first install Lacie RAID Manager on your computer.
This app is the 6big control room. This program has several options for customizing RAID settings, which takes you up to 5 minutes.
LaCie RAID Manager user interface
In the first step, you should create an array (a row of hard drives) that appears as one (or several) storage on the desktop. The process begins by clicking the Create Array button.
RAID Settings Menu
You can configure LaCie RAID Manager to automatically create an array based on existing disks.
There are 6 discs in 6big so RAID 5 is selected. If you are looking for a different level of RAID, click on Custom.
Select the desired RAID level
In this example, the Custom option is selected to select the RAID level. According to Austen, LaCie’s charm of products is their simplicity. You do not need to read the first instructions to use them.
In the screenshot below, you will see the selected RAID level. In this example, RAID 10 is selected, which results in 42 terabytes of memory capacity.
You need to assign a name to your array and go through the next few steps. The name of the array in this example is Monster.
LaCie RAID Manager main window
With a few clicks, all the steps are taken. Now the LaCie RAID Manager main window displays the new Monster Array with its RAID capacity and level. At this point, you can start using the RAID system.
Later you can change many settings, such as alert emails. Austen rarely checks the RAID Manager and sometimes checks only updates.
One of the 5big Thunderbolt 2 systems with RAID 0 is 1050 megabytes per second and 5 rows of drives.
Its other system is the 6big Thnderbolt 3 at 1400 megabytes per second and 6 rows of drives.
The read and write speed of 6big is considerably higher than 5big; the capacity of a laminar TIFF file will be between 2GB and 4GB.
After returning from each trip, Austen arrives at 6big at high speeds, but more importantly, the 6big images are processed directly in Photoshop without any delay.
Austen can continue to work without waiting for the process to process an image in Photoshop and save it.
Opening files also requires a little time. Thunderbolt 3 is the main reason for speed improvements.
Thunderbolt 3 has a transmission speed of 40 gigabits per second, which is twice as fast as Thunderbolt 2 and four times Thunderbolt and USB 3.1; it can only transfer audio, video, data, and even power through a cable.
The LaCie 6big Thunderbolt 3 can easily connect two 4K monitors and several other peripherals to the laptop, power up the laptop, and even charge a camera.
At the back, there are three USB-C ports: 2x Thunderbolt 3 and 1x USB 3.1>
Thunderbolt 3 uses a USB-C connector with a Thunderbolt tag. Since Thunderbolt 3 is protocol-independent, it supports the connection of these interfaces:
USB 2.0, USB 3.0 and USB 3.1, DisplayPort 11 and 1.2a, Thunderbolt Twenty Gigabits per second, Thunderbolt 40 Gbps, and Ethernet (with adapter).
This device is compatible with previous versions. You can use USB 3.1 on your computer, but surely it’s not going to be Thunderbolt 3.
However, by connecting the Thunderbolt 3 iPad 3 (USB-C) to the Thunderbolt 2 adapter, you can connect to the older one at a faster rate.
If you do not have USB-C / Thunderbolt 3, you can use 6Big with the adapter.
Additionally, if you use a laptop to connect to 6big, you can access the array not only through an incoming connection, but you can also charge your laptop at the same time.
Quick video framework
High-speed, ultra-fast Nikon D850 and Nikon Z7 camcorders have captured many movies.
4K video editing and processing is a major challenge for all computers, but the advantage of Thunderbolt 3 is its high speed video processing.
The popularity of the 6big is growing among filmmakers and filmmakers seeking to improve video processing.
Thunderbolt 3, with the advantage of 5big double-bandwidth video, allows you to connect a 4K display or a 5K screen to LaCie 6big.
If you have two 4K screens, you can expand your workspace by assigning a screen to the timeline (video line) and assign another screen for previewing 4k footage.
Two 6big Austen in South Africa, Monster on the left and 48 terabytes copied to the right.
RAID is not a backup
Working with a RAID system to manage valuable data is more confident about working with JBOD (a bunch of disks), but you should keep in mind that RAID is not a backup and even an alternative to backup.
6big, however safe and quick it is, will not protect your data against theft and fire. You must also consider backing up your RAID device as part of your data management.
RAID is a good way to quickly recover hardware failures and can only be part of a comprehensive data recovery solution.
Austen considers another 6big device as a backup outside of his residence and consistently coordinates the second device with the original 6big.
Equipment theft is common among photographers who travel a lot or work on social networks. Social networking makes many people aware of the lack of home photographers at home.
Outdoors outsourcing increases data security against fire and theft
For example, Austen’s residence in the past two years has seen major fires, and many of his neighbors have lost their homes and all their assets.
With proper insurance, you can return more of your assets, but images will be lost forever.
Any backup is better than no backup, but off-the-shelf backup is always better than an in-house backup.
One of the other solutions is the use of slider memory. Cluster memory is a type of computer data memory in which digital data is stored in logical tanks.
Physical memory covers a range of multiple servers in different locations, and the physical environment is owned and managed by the host company.
Cluster memory is a great support for many people. This memory is useful if the number of images is not high, for example, for those who take pictures with the iPhone and spend a lot of time working with the Internet, this type of storage is appropriate.
This method is not suitable for photography like Austen, because he always goes to areas where the signaling of the Internet is weak. In such a situation, loading images on the cloud is not a logical solution.
So for photographers like Austen, investing in backup equipment is essential. As a result, he uses a second 6big device to copy the original version.
Maintenance of the system
Most people put their RAID system on the table. The condition is slightly different from Austen.
His main office is in South Africa, but when he goes to Holland he works on his pictures. Perhaps this is not an ideal situation; but it has been used to many years.
He operates in South Africa on the Mac Pro system and monitors EIZO CG2739 and has two 6big storage systems (one with a mass of 84TB and another with 48TB).
Monster is his original system, and another copy is out of the building, which is connected to the original copy to back up.
If you go to the Netherlands, it will take the 6big system. LaCie designed a special suitcase to protect this system.
LaCie 6big bag
This bag is making by Pelican, so it is completely durable, waterproof, anti-dust and has the necessary resistance to impact, fracture and temperatures below -28 degrees Celsius.
The interior of the foam is suitable for the device cable as well as for the three LaCie Rugged portable drives. There will be no damage in the baggage area at the airport.
On the other hand, there is the possibility of losing this bag at the airport; but this bag is large and large, and the only way to get into the aircraft is to accept baggage.
Austen has also been thinking about this issue. He uses a smaller bag for Pelican, so he can easily take it to the aircraft cabin.
Luggage Peli 1510, in the size of the aircraft cabin
Austen in addition to the 6big can also accommodate the Mac Pro. He says:
When I fill my bag it becomes very heavy. Even the airport staff suspects it, but I show them inside the bag to realize its value.
After reaching the Netherlands, Austin transfers the original information to the 6big backup system. Then it continues with the Mac Pro and EIZO CG247X. He used the same method in the past for the 5big device.
Peli 1510 with 5big and Mac Pro and all cables. The 6big fits easily in this luggage.
The only 6big flaw is its high price. The twelve-tenth version price is $ 1,899, but the price is three times that of $ 3199. For some photographers, the question arises as to what it takes someone to spend $ 3,000 to buy such a device.
They believe they can buy their desired arrays at a lower price, and those who buy 6big pay only the brand money.
Of course, you can build your own array, but one of the benefits of 6big is the disk management software and its 5 year warranty.
LaCie, on the other hand, has a data retrieval service for 5 years; if the problem arises, it will replace the drive and retrieve lost data for free; and if the system is in trouble, it can only be replaced in a few days. .
If you are serious in photography, you should be careful about protecting your pictures. You may spend thousands of dollars on buying a camera and a lens, but if you’re not familiar with the safe way of accessing and protecting information, then your quest will be almost in vain.
LaCie 6big is a good option for people who are determining to manage and protect their data. The 6big is fast, clear, simple, and looks good, and it’s very simple to use.
Instead of buying a new memory every time, it’s best to invest in a suitable RAID system.
If you need more memory, you can replace older drives with larger drives. 6big has a high flexibility and grows together with you. Certainly, in the future,
the size of digital files will increase, and the speed of this growth will increase even more from the past.
For example, in the last few years, the 4K video was a strange choice, but now it has become popular. Multiple cameras have 45+ megapixel sensors.
Austen sees his photos as the most valuable asset and protects them well. In photography tours,
people are often facing with, for example, $ 5,000 to buy a camera and $ 10,000 to spend on buying multiple lenses.
In addition to encouraging photographers to invest in quality triplets and protecting $ 15,000 cameras,
he encourages them to invest in an appropriate data management solution.
The benefits of this include saving time, professional work environment and, consequently,
increasing security levels. To start, you can use small systems and then upgrade them.