Some Hard Truths About Remote Workflow Security
Media and entertainment companies have always struggled to stay one step ahead of security breaches, and the recent increase in remote workflows makes an already difficult task even more challenging. But when we reached out to production and post houses to find out about their approach to remote workflows and endpoint security, we quickly discovered that the first rule of remote workflow security seems to be that you do not talk about remote workflow security. And that’s because there’s a risk that sharing information about how you keep your content secure could be what puts your security at risk… and some folks are worried about revealing just how vulnerable their workflows are.
When we did finally manage to convince a handful of post pros and security specialists to break the silence, they shared some hard truths.
VPN Doesn’t Cut it Anymore
Before the pandemic, virtual private networks were a popular way for off-site staff members to connect to their office network. So when COVID forced post out of facilities and into private homes, a lot of media companies adopted VPNs as a quick fix to enable remote workflows. As accredited TPN assessor and Jigsaw24 consultant Phil Winterhalder explains, “It got very scrappy in the first few months before organizations like the Motion Picture Association and the TPN started to release do’s and don’ts to protect media security.”
But VPNs are far from ideal solutions for media workflows – and not just because they’re inherently slow. “It’s considered a technology solution that just doesn’t give the business enough awareness, enough oversight or control, because once the VPN is established, you can transfer data more or less at your own will,” warns Winterhalder.
Jigsaw24 Media’s head of innovation, Chris Bailey, puts it more plainly, saying, “Providing access via a VPN is literally opening the floodgates to your subnet.”
Instead, Winterhalder and other information security experts recommend providing remote teams with thin client or zero client devices and using pixel streaming or display protocols to access centrally stored files. The limited functionality of thin and zero devices means that users can’t download or screenshot content, while pixel streaming both increases media workflow speeds and improves security because the media doesn’t move from the storage.
“Pixel streaming literally means that the pixels are being sent to you over the internet and then your keyboard and mouse inputs are being sent back – what we’re not doing is sending the actual data,” says Winterhalder.
Security Is Everyone’s Responsibility
There has always been a push-pull dynamic between creative freedom and security in media and entertainment. In the past, getting the job done generally trumped everything else, even when the most stringent protocols appeared to be in place — like the “air-gapped” edit suites that were plugged back into the internet as soon as studio bosses’ backs were turned.
What Bailey describes as the “security Wild West” hit its peak during COVID when, for many, any semblance of secure workflows went out the window in the name of keeping productions running. But the pendulum has swung, and it’s no longer a choice between security and being able to do your job. “Before COVID, security for remote workflows was expensive and not very well-understood. Now everything’s software-defined, internet connections are faster and there are various solutions to choose from, so, from a technology point of view, there is absolutely no excuse, and it really is time to grow up,” Bailey insists. He acknowledges that this might mean some level of inconvenience for users, but he believes the user experience shouldn’t be significantly compromised if media companies implement the right solutions and set up endpoint profiles according to the use case.
It’s a fine balancing act, according to Digital Orchard’s head of technology, Adam Shell, who describes the situation as “a question of keeping your systems flexible and secure and hitting all the guidelines that the studios give you while also allowing your staff to physically be able to do the job that they need to do.”
For Chris Sarson, MD of The Collectv and director of creative remote, the challenge is that the people who use these systems might be fantastic creative editors or producers, but they’re not necessarily the most technically skilled. Another issue is that production teams working under pressure can get frustrated when security protocols change regularly. “All we can do is make sure we’ve got clear information, clear guides and things like that,” Sarson says. “It’s about bringing our production partners, line producers, production managers and post supervisors on board and making them understand that we’ve all got to do what we can.”
There’s No Such Thing as Totally Secure.
Perhaps the hardest truth to accept about information security is reflected in the statement from someone we spoke to who said, “If someone wants to attack you, and they’ve got enough skills, they will get through your system. It doesn’t matter what security is in place.” The fact that the person who made that statement didn’t want to put their name to it demonstrates just how terrifying this prospect can be, but that doesn’t mean you shouldn’t make it as difficult as possible for unauthorized people to access your content.
Sarson’s advice is to never stop working on security: “We’ve got to constantly keep investing, keep looking at ourselves and see what we can improve,” he says. That means some security fundamentals on the userside are non-negotiable. “You have to use two-factor authentication, even if that’s tricky for some people,” he insists. There’s good reason for Sarson’s hard line on 2FA, as Winterhalder confirms: “In this industry it’s just considered mandatory; no studio is going to accept a remote access solution that doesn’t use two-factor authentication.”
If 2FA is standard practice, then zero trust methodology takes this principle to the next level. Winterhalder explains that zero trust means not assuming you’re an authorized user – even if you get through the two-factor authentication process, and especially if there are any unusual circumstances. He says, “Zero trust is about building up a set of rules and conditions for access and not trusting anyone unless they meet those criteria. So, if you’re connecting on a Saturday when you normally only work during the week, or you dial in from Germany instead of France, or you use a different laptop, you might be denied access or have to complete additional verification processes.”
But one of the biggest security risks for remote workflows is the users. While organizations can control physical access in facilities, privacy regulations prevent companies from prescribing work setups in homes, so organizations like the TPN can only recommend best security practices for remote workflows. Many of these remote security protocols – like ensuring that your screen isn’t visible to other people or locking your device before stepping away – rely on user cooperation, which can never be guaranteed. And that’s assuming that the user isn’t complicit in leaking your content. Bailey points out, “Remote security is always at risk of the user – if they’re not on-site, they can always point a phone at the screen and hit record.” This is where watermarking comes into play. Visible watermarks embedded in the media are often used to deter people from using copyrighted content, while forensic watermarks are invisible and serve to track the origin of content leaks.
Not All Content Is Created Equal
The good news is that not all content needs the same level of security. Bailey recommends implementing workflows that flux according to the content type and the associated risk level. As he puts it, there’s no point implementing the same protocols for Apple’s product launch material and Homes Under the Hammer.
Digital Orchard adopts this tiered approach for its projects, which range from dailies for high-end TV and studio shows to post production for small independents. “Our dailies department is entirely air-gapped so that there’s no way you can access studio material without physically being in the building, but the security specs for our post work are not nearly as restrictive,” Shell says. “It’s about being able to meet the levels of demand from different clients and having the flexibility to move between those levels. We’d love to have remote access for everything, but without an extremely knowledgeable and proactive IT team to manage that process, we would open ourselves up to a considerable amount of risk.”
The concept of air gapping may not translate directly to remote workflows (which are inherently connected), but the principle does apply to how media companies should break down remote access to their network, according to Winterhalder. He describes how network hierarchies should be set up with the most untrusted network (the internet) at the top and each layer below that being increasingly secure — from your DMZ to corporate and production networks.
“You should never be able to move directly from an untrusted network, like the internet, to a secure network where your content resides without using a broker or relay through a second connection,” says Winterhalder, adding that breaking access down in this way creates something akin to an air gap and ensures that a security compromise should only affect the first layer before it’s caught. But network architecture of this nature is best left to the IT professionals and might not be necessary for every media company.
Who Picks Up the Bill?
The main excuse for skimping on remote workflow security is cost – particularly when it comes to boutique post houses and the “race to the bottom” for offline editing which can result in risky shortcuts. Shell describes the dilemma Digital Orchard and many other companies face. “As we grow, we will need to employ or bring in outside consultation to look after security for our systems. But we will have to try and keep things simple so it can be managed by as few people as possible because it’s not something that generates revenue.”
On the other hand, Sarson is emphatic about the need for change in offline editing: “It’s very simple. Security costs money. We have it with our policing and in day-to-day society — a certain amount of budget has to go toward security, and it’s the same for our media systems,” he insists. “If the content that we’re creating is really that valuable, then we need to put a proportional cost of the budget into security. That means that the price of offline cannot keep going down to almost nothing.”
While it’s unlikely that remote workflows will ever be 100% secure, end-point security has clearly come a long way in the past few years. And the technology keeps getting better.