Bigdata Assignment
Aims
The aim of this assignment is to introduce a practical application of Big Data and Cloud Computing using a realistic big data problem. Students will implement a solution using an industry leading Cloud computing provider together with the distributed processing environment Apache Spark. This will involve the selection of problem appropriate Machine Learning algorithms and methods.
Learning Outcomes Assessed:
Knowledge & Understanding
- Apply big data analytic algorithms, including those for visualization and cloud computing techniques to multi-terabyte datasets.
- Critically assess data analytic and machine learning algorithms to identify those that satisfy given big data problem requirements
Intellectual / Professional skills & abilities:
- Critically evaluate and select appropriate big data analytic algorithms to solve a given problem, considering the processing time available and other aspects of the problem.
- Design and develop advanced big data applications that integrate with third party cloud computing services
Personal Values Attributes (Global / Cultural awareness, Ethics, Curiosity) (PVA):
- Critically assess the relationship between knowledge and the ethical and social interpretation of primary research using big data.
Definitions
Portfolio Assignment: A collection of pieces of work
Individual Work: Work carried out by one person only
Group Work: Work carried out collaboratively seeking to improve each other’s elements
Peer Review: Critical analysis and subsequent grading of a social equal’s work
Semi-Formative: Training tasks assigned course credit to reward and ensure engagement.
Assignment Overview
The portfolio assignment is divided into components as follows:
| Training Tasks (30%) |
Semi-formative elements of the portfolio constitute 30% of the assessment for this module and include, group, individual, and peer assessed work |
| Combined Big Data Product and Report: (70%) |
Individual work – Combined Big Data Product and Report:
|
Training Tasks
Training Task 1: Peer Reviewed Task (24%)
The objective of this task is to ensure that students have mastered these skills which are required for final module assessment:
- Process a data set using the recommended software environment for the module.
- Explaining the logical reasoning behind your code.
This work will be peer assessed as recommended the British Computer Society. That is, you will critically assess the work of fellow students (your peers) and THEY will assess yours.
In detail:
- You will create a Jupyter notebook based the scenario below (which is derived from weekly worksheets 1-4) explaining your code using notebook embedded Markdown (i.e. formatted text, not just comments)
- You will post your notebook to the module discussion board on Blackboard
- You will then mark (i.e. peer review) the submission preceding yours on the discussion board, and the one following it, using the marking scheme below and post these mark sheets
- Your mark for this task will be the average of your peer marks.
Scenario:
Suppose you are a police department with a limited budget. You plan to reduce road-traffic accidents by a one-month targeted advertising campaign.
Using the given dataset, which gender, age group, and month would be the largest target group as indicated by positive breath tests?
Peer Reviewed Task Marking Scheme
| The following marking scheme will be used for this assignment. A downloadable form will be placed on Blackboard that leaves space for comments. You should apply the general Marking Criteria to scale each element. This will be discussed in class. | |
Description |
Marks |
| Peer Reviewed Task Marking Scheme: | (100) |
| Introduction: The targeted advertising campaign task and approach taken to the problem | 10 |
| Component Selection and Data Pipeline Implementation (code) | 20 |
| Data Extraction and Filtering System running, test and diagnostics | 20 |
| Design, Development and reasoning behind visualization methods, and statistics | 20 |
| Detailed Analysis and consideration of the appropriateness of the solution for the initial problem | 10 |
| Evaluation and Conclusion | 10 |
| Style and Referencing | 10 |
| Total Marks Available: | 100 |
Notes:
- Staff reserve the right to adjust marks that are disproportionately high or low, or where the two reviews differ by more than 20%.
- Total marks from the two reviewers will be scaled to 24% of the module mark.
