Loading...

What We Do

/What We Do
What We Do2018-10-30T12:00:35+00:00

PROJECT DELIVERABLES

ID Title Issued Link
WP1: Project Management
D1.1 Project Management Manual September 2015 PDF File Logo
D1.2 Project Quality Assurance Manual September 2015 PDF File Logo
D1.3 Project Risk Management Plan November 2015 Confidential
D1.4 Data Management Plan November 2015 PDF File Logo
D1.5 SWOT Analysis February 2016 Confidential
D1.6 First Year Annual Report October 2016 Confidential
D1.7 Data Control Report February 2017 Confidential
D1.8 Second Year Annual Report September 2017 Confidential
WP2: Data Protection, Privacy, Ethical and Criminal Law (DaPPECL) Constrains
D2.1 Report on the Data Protection, Privacy, Ethical and Criminal Law (DaPPECL) Frameworks February 2016 PDF File Logo
D2.2 Framework for Impact Assessment of FORENSOR against DaPPECL Requirements May 2016 PDF File Logo
D2.3 The Impact Assessment Report (restricted and public versions) August 2016 PDF File Logo
D2.4 First Report on Monitoring of Observance of DaPPECL Requirements March 2017 Confidential
D2.5 Report on Potential Misuse Implications and Risk-mitigation Strategies February 2017 Confidential
D2.6 Report on Potential Dual Use Implications and Risk-mitigation Strategies August 2017 Confidential
WP3: Requirements, Specifications and System Architecture
D3.1 Use Case Analysis and User Scenarios February 2016 Confidential
D3.2 System Requirements and Architecture Design Document June 2016 Confidential
D3.3 System Requirements and Architecture Design Document – Final November 2016 Confidential
WP4: Ultra-low-power Vision Sensor Development
D4.1 I Vision Chip Tape-out June 2016 Confidential
D4.2 Report on the Design and Experimental Results of the 1st Vision Chip January 2017 Confidential
D4.3 II Vision Chip Tape-out August 2017 Confidential
D4.4 Report on the design and experimental results of the 2nd vision chip October 2018 Confidential
WP5: Ultra-low-power Vision Algorithms
D5.1 Algorithms (Low- and High-level) Interim Report August 2016 Confidential
D5.2 Algorithms for High-level Scene Analysis and Event Detection, Preliminary Version February 2017 Confidential
D5.3 Embedded Low-level Algorithms May 2017 Confidential
D5.4 Final report on ultra-low-power vision algorithms November 2017 Confidential
WP6: FORENSOR Secure Communication Infrastructure
D6.1 Network Topology Design and Evidence Sharing Mechanisms November 2016 Confidential
D6.2 Security Measures, Implementation and Benchmarking – Initial December 2016 Confidential
D6.3 Routing Criteria, Protocol and Mechanism Specification, Implementation, Evaluation – Initial February 2017 Confidential
D6.4 Monitoring and Visualization February 2017 Confidential
D6.5 Security Measures, Implementation and Benchmarking – Final August 2017 Confidential
D6.6 Routing criteria, protocol and mechanism specification, implementation, evaluation – Final November 2017 Confidential
D6.7 Notification and control application October 2017 Confidential
WP7: System Integration and Validation
D7.1 Integration Strategy and Planning June 2016 Confidential
D7.2 FORENSOR Platform, 1st Release February 2017 Confidential
D7.3 FORENSOR platform, 2nd release November 2017 Confidential
WP8: Pilot Plans and Evaluation
D8.1 Report on Data Preparation and Acquisition Procedure February 2017 Confidential
D8.2 Pilots and Field Trials Planning February 2017 Confidential
WP9: Dissemination and Exploitation
D9.1 Project Website November 2015 webicon
D9.2 FORENSOR Market Analysis Report August 2016 Confidential
D9.3 FORENSOR Business Plan November 2016 Confidential
D9.4 Report on Dissemination Activities February 2017 PDF File Logo
D9.5 1st Workshop Report December 2017 PDF File Logo
D9.6 Initial FORENSOR exploitation plans October 2017 Confidential

SCIENTIFIC ARTICLES

Journals

No. Citation Link OA Link
1 Böröcz, István. (2016). Risk to the Right to the Protection of Personal Data: An Analysis Through the Lenses of Hermagoras. European Data Protection Law Review, 2(4), 467–480. http://doi.org/10.21552/EDPL/2016/4/6 Closed Access logo Open Access logo

Conferences

No. Citation Link OA Link
1 Y. Zou, M. Gottardi and M. Perenzoni, “Live Demostration: Low Power Vision Sensor with Robust Dynamic Background Rejection,” 2018 IEEE International Symposium on Circuits and Systems (ISCAS), Florence, Italy, 2018, pp. 1-1. Closed Access logo
2 Y. Zou, M. Gottardi and M. Lecca, “A 1.6mW 320×240 pixel vision sensor for event detection,” 2017 International SoC Design Conference (ISOCC), Seoul, 2017, pp. 97-98. Closed Access logo
3 Y. Zou, M. Gottardi, D. Perenzoni, M. Perenzoni and D. Stoppa, “Live demonstration: Motion detection vision sensor with dynamic background rejection,” 2017 IEEE SENSORS, Glasgow, 2017, pp. 1-1. Closed Access logo
4 Y. Zou, M. Gottardi, D. Perenzoni, M. Perenzoni and D. Stoppa, “A 1.6 mW 320×240-pixel vision sensor with programmable dynamic background rejection and motion detection,” 2017 IEEE SENSORS, Glasgow, 2017, pp. 1-3. Closed Access logo
5 G. Zoumpourlis, A. Doumanoglou, N. Vretos and P. Daras, “Non-linear Convolution Filters for CNN-Based Learning,” 2017 IEEE International Conference on Computer Vision (ICCV), Venice, 2017, pp. 4771-4779. Closed Access logo Open Access logo
6 M. Lecca, Y. Zou and M. Gottardi, “A low power smart camera for video surveillance and forensic applications,” 2017 International Conference on Engineering, Technology and Innovation (ICE/ITMC), Funchal, 2017, pp. 626-631. Closed Access logo
7 A. Doumanoglou, N. Vretos and P. Daras, “Action recognition from videos using sparse trajectories,” 7th International Conference on Imaging for Crime Detection and Prevention (ICDP 2016), Madrid, 2016, pp. 1-5. Closed Access logo Open Access logo

FORENSOR OVERALL CONCEPT

FORENSOR Architecture

The FORENSOR system.

The overall concept of FORENSOR is summarized in the figure above. At the heart of FORENSOR is the use of a new System-on-Chip (SoC) developed by Partner STMICROELECTRONICS SRL. Their current SoC will be enhanced to reflect the specific needs of FORENSOR in terms of security and functionalities. The existing SoC is implemented using cutting edge paradigms for ULP computation with parallel processing, composed by multiple ULP DSP blocks and specific HW accelerators. The combined computational capacity for specific tasks, is equivalent to GHz level leading edge ARM processors, but at 2 orders of magnitude lower power. The new implementation of this SoC will support integration of several functionalities; integrated either in silicon, or as algorithms but in any case implemented following a joint approach to ULP computation. The set of functionalities of FORENSOR is as follows:

Current commercial imagers are almost always designed for multimedia applications, mobile phones, digital video cameras and toys; low cost and high image resolution and quality are the figures of merit. In the literature, several examples of custom vision chip implementations are reported, aimed at low-power applications. However, the majority of them typically confront only the sensor perspective, without implementing scene recognition algorithms. In FORENSOR, we will develop into an operational FORENSOR device a prototype imaging sensor (developed by Partners FONDAZIONE BRUNO KESSLER and EMZA VISUAL SENSE LTD) that addresses the requirements of both ULP and integrated image pre-processing capability (motion detection, background subtraction, etc.). We will target custom digital electronics to be interfaced/integrated with the main processing platform/SoC for further digital processing to be executed by the DSP and hardware accelerators.
Following strict constraints of power budget associated with an autonomous sensor, we will develop and optimize low-power image processing algorithms adapted for execution on the main processing platform/SoC. Significantly facilitated by pre-processing of image performed on the imaging sensor array (as described above), the developed algorithms should i) operate on very low bpp (bits per pixel) input; ii) exploit the low-power processor of the platform; iii) operate intermittently (only when required) and iv) provide sufficient performance and accuracy to make the sensor useful for the evidence gathering requirements of the FORENSOR use cases.
The FORENSOR sensors will be equipped with ultra-low-power wireless transceiver chip that enable secure mesh networking scenarios. Networking allows the distribution of the information gathered and interpreted by the system among peers and towards the remote monitoring and control centre for real time notification and control. Security will be safeguarded through encryption methods and secure-by-design routing algorithms. In terms of energy conservation, the communications duty cycle will be restricted; we estimate 99% of the time in deep sleep. Furthermore the intelligent, routing algorithm will allow for energy awareness. While we rely on off-the-shelf radio modules, we will target improvements on robustness, security and energy efficiency exploiting the node intelligence and adapting state-of-the-art clustered mesh networking protocols, for integration and execution on the main processing SoC/Platform.
The ULP vision chip described above will work in ultra-low-power mode and will be able to automatically detect an abnormal activity using the embedded low-level image processing algorithms. Only when an abnormal activity is detected, the more power-hungry operations of high-level scene interpretation and communication or storage of video segments will take place. In case the resolution of the ULP vision chip (QVGA or VGA) is not adequate for the specific use case (e.g. cannot provide distinguishable characteristics of people or vehicles to support evidence), an additional high-resolution vision sensor will be also integrated into FORENSOR platform. Existing commercial sensors will be used in this case.
FORENSOR aims to create a platform that will gather and deliver evidence for criminal prosecution that will be accepted in a court of law. In order to ensure acceptance, there are requirements for both the hardware and the software of the platform. The hardware parts of the platform should be certified to warrant the reliable operation of the sensor. However this will be performed after the finalization of the product (TRL8). On the software side, FORENSOR will handle evidence gathered in a transparent way to ensure traceability of any processing performed. Furthermore, FORENSOR will log the recording metadata and any activity on the extracted evidence, and will provide this information to the defendor for examination.
The power consumption of the FORENSOR platform will be determined by the balance of continuous operation of the ULP sensor and a basic ULP video processing to detect possible threats, the occasional use of high resolution sensors and full processing power to analyse the scene, the activation of data storage for evidences collection and the communication channel used only when some detected and verified threat should be communicated. The power manager will take care of these elements in order to assure a long duration of the batteries of the device, providing a substantial autonomy to the overall system reducing the need of frequent maintenance.

ULP VISION CHIP EXPLAINED

Operations of the Ultra Low Power (ULP) Vision Chip with Embedded Image Processing: The two videos below show the type of image processing executed by the visual sensor to extract salient visual features needed to detect potential alert situations. Such processing needs to be very low-power, being continuously computed by the sensor (100% of the time). The output of the sensor is binary (hot-pixel bitmap and xy projections) and is delivered to the external processor only if the chip detects an alert. Therefore, its duty cycle is expected to be much less than 100%. Considering that data delivering is a power hungry operation, this will turn into a significant reduction of the power consumption.

 

Operations that are continuously executed by the low-power vision chip:

  1. Image acquisition;
  2. Hot-Pixel detection;
  3. Hot-Pixel noise removal;
  4. Hot-Pixel xy projections, used to detect alert situations.

 

Data delivered by the sensor to the external processor only in case of alert:

  1. Hot-Pixel binary map and xy projections.

 

Operations that are continuously executed by the low-power vision chip.