Summary Report
Co‐Pilot Needs Assessment
Table of Contents Executive Summary ...................................................................................................................................................... 3 Overview of Main Findings .......................................................................................................................................... 4 High level overview of the results. ............................................................................................................................ 4 Trainer Need ................................................................................................................................................................. 5 Determining Co‐Pilots Focus ..................................................................................................................................... 5 Quick & Easy Setup Interface .................................................................................................................................... 6 Censorship Profile Sharing ........................................................................................................................................ 6 Censorship Features ..................................................................................................................................................... 7 Configurable to a variety of different environments ................................................................................................ 7 Domain name system (DNS) filtering and redirection .............................................................................................. 7 URL (Uniform Resource Locater) filtering ................................................................................................................. 7 Hardware Considerations ............................................................................................................................................ 9 Core Hardware Specifications ................................................................................................................................... 9 Number of Supported Clients ................................................................................................................................... 9 Device evaluation based upon survey results. ........................................................................................................10 Conclusions: Where do we go from here? .................................................................................................................12 Follow‐up Interviews ..............................................................................................................................................12 Iterative Design .......................................................................................................................................................12 Appendices .................................................................................................................................................................13
2
Executive Summary
Internews is building a prototype "Training Co‐Pilot:" an easy to use censorship simulating access point in a box. The Co‐Pilot will allow a trainer to simulate various censorship environments so that trainees are able to safely and actively experience using circumvention tools in a censorship environment. The Co‐Pilot team conducted an initial needs assessment to ensure the prototype addresses the core issues digital security trainers face in the field. The survey was sent to 30 members of the digital security training community and 20 responses were received.
The main goals of the survey were two‐fold: 1. to better understand the current environment trainers work in with their participants; 2. to collect and receive feedback from trainers regarding what they would like to see in a training aid such as Co‐Pilot.
The feedback collected from the training community will directly impact the design and creation of the Co‐Pilot. The initial questions in the survey not only provided background information for the Co‐ Pilot project, but also gave the developer valuable information in the back‐end infrastructure design. The information also provided an overview of the current training environment and highlighted specific issues trainers face in the field.
3
Overview of Main Findings
The trainers who completed the survey had a variety of important features for Co‐Pilot, but two themes stood out. Co‐Pilot must be.... • Easy to setup and use. When asked what would cause a trainer to use, or not use a tool like co‐pilot the majority wrote that "ease of setup and use" was critical. • Configurable to a variety of different environments. Trainers want to be able to create, and switch between, custom censorship environments.
High level overview of the results.
Trainer Interface Features
• Quick & Easy Setup Interface • Censorship Profile Sharing
Censorship Features
• Configurable to a variety of different environments • Domain name system (DNS) filtering and redirection • Uniform Resource Locator filtering
Hardware Considerations
• • • • •
Open‐Source Sub 100$ device Not Suspicious Looking Pocket Sized Light Weight
4
Trainer Need Determining Co‐Pilots Focus
During the survey, trainers were asked if they would use a tool like Co‐Pilot. The team was curious if trainers would feel this type of tool would useful for assessment of their participants’ retention and understanding after the training, or in providing deeper understanding and further explanation of the topics being discussed.
Despite the high “yes” response rates the comments from this question contained some critical feedback. Two trainers stated that it was really not necessary for participants to understand circumvention at this level – they just need a tool that works and works reliably. This pointed to a need for a sub‐set of trainers who simply want co‐pilot to verify for participants that a tool is working, not necessarily a way in providing explanation as to how that tool works. In response to these comments the Co‐Pilot team has decided to focus on making an easy to use tool for training users on the proper use of circumvention tools. As such, we are not creating any “user‐facing” interfaces or educational content to go along with the Co‐Pilot tool at this time
5
Quick & Easy Setup Interface
When asked what would cause a trainer to use, or not use a tool like co‐pilot the majority wrote that ease of setup and use was critical. While not entirely unexpected, this was a major theme in all questions that explored adoption by trainers.
Understanding that ease of use was one of the most important features of Co‐Pilot, multiple iterations of usability testing are planned to happen throughout the development process – with both trainers and training participants. Usability testing will occur not only in Washington DC, but also at the Circumvention Tech Festival in Valencia, Spain, where
Most Important Features “Ease of set‐up and guidance on real‐ world scenarios where such blocking could be in place and guidance on advising which tools can overcome the blocking …” “Ease of setup and reliability. Good supporting documentation.” Selective ability to turn different types of blocking on and off without being a hassle.”
Trainers Summit of 60 digital security trainers from “An easy‐to‐use interface for trainers.” around the world will be convening. Additional user interface testing will happen at RightsCon in Manila, where Co‐Pilot will be showcased in the demo room while continuing to collect user feedback.
Censorship Profile Sharing
One of the benefits of having a wide range of digital security trainers take the survey is being able to see its applicability in a variety of different settings and contexts around the globe. Understanding the value trainers from different regions and contexts see in this type of tool, also has implications for the tool’s design.
For example, many trainers mentioned specific countries when talking about demonstrating censorship environments. This has brought about internal discussion on how to incorporate country‐specific censorship and blocking into the Co‐Pilot prototype. Given that country‐ specific censorship is difficult to track and takes a significant amount of resources to stay up‐ to‐date, another option that has been explored is the ability to share different blocking configuration profiles with other trainers. It would then allow trainers to pool resources, while also leverage the existing digital security and tech community research that is occurring. Support for this possible future use‐case is seemingly easy to implement if it is part of the design process from the beginning. The Co‐Pilot team will be implementing the ability the save and load profiles from a USB drive in the initial prototype. Beyond this, the team will be adding optional descriptions for profiles created so that people can describe the function or narrative of a censorship profile when they are sharing it.
6
Censorship Features
In the survey we attempted to narrow down the types of circumvention tools that would most benefit from Co‐Pilot. The survey asked about the different circumvention tools most commonly trained on and which of those tools would be most useful for Co‐Pilot. Knowing this information affects the different types of blocking configurations that should be made available in the tool.
As seen by the first graph above, trainers train nearly as much on tools that only support web browsing (Tor Browser Bundle, Orweb) as tools that provide more broad circumvention (VPN's, Orbot). Yet, when asked which tools would benefit the most from Co‐Pilot VPN's took the definitive lead.
Configurable to a variety of different environments
Having a variety of configurable censorship environments, the Co‐Pilot developer will work to make Co‐ Pilot customizable both for the tech savvy trainer and the trainer that does not have as technical a background. To support configurable profiles the Co‐Pilot team has decided to break apart censorship profiles into sub‐components that are easy to customize and to allow any censorship profile to be customized while it is being used.
Domain name system (DNS) filtering and redirection
When asked which types of hostile digital environments would benefit the most from a Co‐ Pilot like environment. The highest response from trainers indicated they wanted HTTPS certificate spoofing and DNS redirection – both active manipulation of the network – while passive blocking/throttling methods closely followed.
Based upon this feedback Co‐Pilot will be implementing a set of “Domain name system (DNS) filtering and redirection” capabilities during the prototype phase.
URL (Uniform Resource Locater) filtering
Most users of website browsers use the human readable URL (Uniform Resource Locater) when they 7
wish to access a website. With the heavy use of website based circumvention tools being trained on, the team decided to make URL based censorship an initial feature of Co‐ Pilot. This will allow Co‐Pilot to support censorship that trainers can easily show by asking users to connect to these URL's via their browsers. The two graphs below illustrate the different circumvention tools trainers are using in their trainings and which would be most useful for the Co‐Pilot.
8
Hardware Considerations
Hardware specifications were derived from the answers given on questions about what would prevent trainers from using a piece of hardware like Co‐Pilot in their trainings.
Core Hardware Specifications The core hardware specifications were teased out through survey feedback to questions such as: What would prevent you from using an environment like this for your trainings? • Many responses reflected that weight was factor; • In addition, because of the travel trainers regularly are required to do, many were concerned about the look of the device itself and ensuring it is not appear suspicious. What would prevent you from using an environment like this for your trainings? • A proprietary device. • It would be nice to be able to run this off of a Raspberry Pi or another sub‐100 USD SoC. • Form factor is an interesting issue. If it fits in the pocket, it's a no‐brainer, if it's a big ol' Linksys cigar box, can't always roll with that on board. Based on the feedback received we were also identified some key factors for identifying hardware, such as: • Open‐Source • Sub 100$ device • Not Suspicious Looking • Pocket Sized • Light Weight Making the device “not suspicious looking” is very much a matter of case design. As such the team will be evaluating the original proposed 3d printed cases and off‐the‐shelf cases to see which ones are less likely to draw attention to themselves.
Number of Supported Clients The Co‐Pilot device must be able to support a max of 15 clients and be powerful enough to support at least 10 users traffic comfortably. The survey revealed that almost half of the trainers tend to have more people in their training session than they would ideally like. This highlights the current high demand and small number of digital security trainers. The majority of trainers (85%) preferred 10 participants or less in any sort of hands‐on training session, when in actuality almost half of the trainers tend to have more than 10.
9
Supporting 15 clients means that we need to prototype on multiple devices with a variety of levels of CPU, GPU, and memory so that we can identify the requirements for running the Co‐Pilot platform for this identified number of users.
Device evaluation based upon survey results. To identify hardware that meets the specifications created by the survey, the team began a research process that led to the following set of possible devices.
10
Open‐Source Software support
Name Bananna Pi Beagle Bone Black
Open‐Source Hardware
Sub 100$ device
Pocket Sized 1
Wei ght
48g
40g 119 g
MinnowBoard
ODROID‐U3
2
78g
HummingBoard
3
48g
Based upon the information that was gathered, the team decided to deploy the Co‐Pilot system on at least two platforms during the prototype phase; the ODROID‐U3 and the Beagle Bone Black.
1 2
Pocket sized was determined as 3 12 x 5 1/2 inches or 89 x 140 mm. It was an embarrassing long search before the writer realized that there is not a standards organization in the world who would ever care about specifying pocket sized. So, I used the "pocket-sized" moleskin notebook as the guide. Even though the name ‘Odroid’ is a portmanteau of ‘open’ + ‘Android’, the hardware
3
isn't actually open because some parts of the design are retained by the company. "We don't supply/sell any PCB design file or Gerber file. Please don't ask about it." For linux to be used on the humming-board a forked kernel is recommended, and a BLOB is required for GPU use
11
Conclusions: Where do we go from here? Follow‐up Interviews
Follow‐up interviews will occur with a select number of trainers who would like to provide additional feedback on the project. These interviews will assist the developer in producing and modifying both the back and front‐end development of the tool by further exploring additional features and tool enhancements. As more trainers are given the opportunity to provide their ideas and experiences in the field, Co‐Pilot will better be able to adapt to the specific needs of the digital security trainer community from different regions and working with different target audiences.
Iterative Design
Development of backend processes will continue in parallel with the user interface design. Interviews with trainers and user interface experts will continue through different stages. The initial stage will be reviewing wire frames of the prototype through “think aloud testing” at the conference held in Valencia, Spain. This will provide the Co‐Pilot team with a unique opportunity to speak with trainers from over 22 countries – training on a variety of different groups (e.g. journalists, human rights activists, LGBT community, etc.) about the project, and ask them to help direct the functionality to be the most useful for these trainers.
All of this is in preparation for the conference in Manila where the tool will be showcased in the demo room. In addition, front‐end user feedback will also be collected at the event and then incorporated into the final prototype
12
Appendices
Appendix A: Problem Definition Mapping Appendix B: Co‐Pilot Survey Questions
13
Appendix A: Problem Definition Tool: Problem Definition Description: particularly effective when trying to focus a team on the key problems at hand; introduces a small set of key criteria by which an issue can be articulated and assessed (enables efficiency); Goal: designed to structure the analysis of a particular problem in a way that makes good use of time; standardized way to compare several different problems which might seem to be very different on the surgace
What is the issue you are trying to address? Why is it important?
Who is it a problem for?
What social/cultural factors shape this problem?
issues are abstract and we need to make them more concrete
trainer tool NOT end-user tool
trainers are PMs - will not do anything that will disrupt training
consequences of not using tool or using incorrectly may not obvious (encryption, circumvention)
trainer tool NOT end-user tool
do not want to necessairly learn all the tech
usually trainees are using tools for 1st time in potentially dangerous scenario
non-tech trainers have lots of difficulty with setting up virtual environments and even some technical trainers have trouble or logistically it is difficult
$Cost$ (the fact that it costs money is a problem)
end-users do not know which tools to use in different situations
implications for users!!!!!
trainers constantly are traveling (must be very portable)
trainers can't tell if they were successful
language (must be multi-lingual)
not all trainers are able to setup virtual environments to illustrate these concrete examples and simulations
tool standards in the community - must be recommended and vetted by other trainers (DJ!!!); adoption model
IMPORTANCE: affects learning of tools and concepts; not understanding or not being able to practice before they actually are in a situation where they need to use the tools affects ADOPTION
What evidence do you have that this is worth the investment?
Can you think of this problem in a different way? Can you reframe it?
engine room reports (metrics for assessing success; training resources for organizing trainings)
trainers
anecdotal from trainers
users
rates of training of trainees
technologists
education methodology (best way people learn is by doing) SURVEYS will provide evidence
Survey Goals (see notes 11/12) type of censorship that would be most useful to demonstrate in training environment how they would incorporate into training? tool focused/ concept focuesd
Appendix B: Co-Pilot Survey Questions
CoPilot Trainer Survey Internews is prototyping a Training "CoPilot:" an easy to use censorship simulating access point in a box. The CoPilot will allow a trainer to simulate various censorship environments [Blocking Tor, Throttling psiphon, Blocking IPAddress', Blocking Traffic by keywords] so that trainees are able to safely and actively experience using circumvention tools in a censorship environment. The core output of this project will be a prototype of an easy to use, small formfactor, open source tool that will allow a digital security trainer to simulate a variety of hostile digital environments, such as a censoring firewall. With your insight we hope CoPilot will improve the quality and effectiveness of digital security trainings. We hope the interactive and safe "hostile" digital learning environments CoPilot will provide in turn will contribute to a greater degree of confidence and positive behavior change for trainees. The following questions will help guide Internew's design and development of this prototype. * Required
1. Name (OPTIONAL)
2. Contact info (OPTIONAL) An email, skype adress, jabber address, etc. we can contact you with.
3. Can we contact you with follow up questions? * We may ask some survey takers to take part in follow up interviews. We may also have questions about somthing you wrote. Mark only one oval. Yes No 4. Do you train on circumvention technology? * (e.g Tor, VPN's. PGP. Cryptocat) Mark only one oval. Yes No
Skip to question 22.
Training Cont.
5. What tools do you do training on? * Please check all that apply. Check all that apply. alkasir Tor Browser Bundle Orweb Psiphon Proxies VPNs Orbot Lantern I2P Ultrasurf Other:
Hands on Training 6. Do you do hands on training with any of these tools? * Mark only one oval. Yes No
Skip to question 16.
Hands on Training Cont. 7. What tools do you do hands on training on? Please check all that apply. Check all that apply. Orweb VPNs Psiphon Proxies alkasir Orbot Ultrasurf Lantern I2P Tor Browser Bundle Other:
8. How many people do you normally have in the classroom for a handson training? Mark only one oval. 15 People 510 People 1015 People 1520 People 9. What is the ideal number people in these types of training's? Mark only one oval. 15 People 510 People 1015 People 1520 People 10. On average how long do you spend with handson training vs. explanation of the tool? Check all that apply. 5% of the training is hands on. 10% of the training is hands on. 25% of the training is hands on. 50% of the training is hands on. 75% of the training is hands on. 100% of the training is hands on. 11. How do you provide trainees the tools they are being trained on? Check all that apply. Official Tool Website USB CD/DVD Local Wireless Download (Library/Pirate Box) Personal Website / Cloud Service (Dropbox, Personal Website)
The "How" of Curcumvention Technology
12. On average do you know if your trainees can successfully use the tool after a training? * Mark only one oval. Yes No Other: 13. If so: How do you know if trainees can successfully use the tool after a training? (e.g. Surveys, have them access hidden servicess, watch them use the tool?) 14. Do you assess the confidence of a trainee in using a tool after/during a training? * Mark only one oval. Yes No Other: 15. If so: How do you assess the confidence of a trainee in using a tool after a training? (e.g. Surveys, interviews, followup emails, conversations?)
The "Why" of Circumvention Technology 16. Do you feel you are able to articulate the importance of circumvention technology in your trainings? * Check all that apply. Never Sometimes Usually Always
17. How long do you spend on explaining the importace of circumvention vs. training on how to use circumvention technology? Check all that apply. 5% of the training is on the importance. 10%of the training is on the importance. 25% of the training is on the importance. 50% of the training is on the importance. 75% of the training is on the importance. 100% of the training is on the importance. 18. How do you explain the importance of circumvention technology? 19. Are there any challenges to expressing the importance of these circumvention tools? Please describe those challenges here.
The "When" of Curcumvention Technology 20. Do you assess if trainees can identify when they need to use circumvention tools? * Mark only one oval. Yes No
21. If so: How do you assess if trainees can identify when they need to use circumvention tools? (e.g. Play Acting, surveys, interviews, followup emails, conversations?)
CoPilot & Its Role in Training Internews is prototyping a Training "CoPilot:" an easy to use censorship simulating access point in a box. The CoPilot will allow a trainer to simulate various censorship environments [Blocking Tor, Throttling psiphon, Blocking IPAddress', Blocking Traffic by keywords] so that trainees are able to safely and actively experience using circumvention tools in a censorship environment. The following questions will help guide Internew's design and development of this prototype. 22. Would you consider using a tool like this to assess trainee’s ability to use specific tools? (e.g. Blocking specific sites locally to allow users to experience success circumventing censorship with tools such as Tor) Mark only one oval. Yes No Other: 23. If no, why not? 24. If yes, what would be the most important feature that you would need to use a tool in this manner.
25. What would prevent you from using an environment like this for your trainings? 26. What tools would benefit the most from an environment like this? Please check all that apply. Check all that apply. Proxies Orweb Ultrasurf None Psiphon Lantern Orbot alkasir I2P Tor Browser Bundle VPNs Other: 27. Would you consider using a tool like this to increase participants understanding of a topic. (throttling, dns redirection, https blocking, application blocking/throttling) Mark only one oval. Yes No 28. If not, why not?
29. If so, what would be the most important feature that you would need to use a tool in this manner? 30. What would prevent you from using an environment like this for your trainings? 31. What topics would benefit the most from an environment like this? Check all that apply. URL based blocking / throttling DNS based blocking / throttling HTTPS based blocking / throttling HTTPS Certificate Spoofing DNS redirection Application based blocking / throttling Other: 32. Of all the trainings you do, where could you see a tool like this being the most valuable? Example: “It would be most valuable with USbased journalists who do not see censorship as an issue whatsoever” or “It would be the most valuable when training on how to distinguish censorship from connectivity issues.”
Powered by