ICA-CIS4035-N-CT20-ADS IN-COURSE ASSESSMENT (ICA) SPECIFICATION Module Title:Machine LearningModule Leader:Dr. Alessandro Di StefanoModule Code: CIS4035-NAssignment Title:Machine Learning Application and ReportDeadline Date: 14/05/2021Deadline Time: 4:00pmSubmission Method:Online (Blackboard) Middlesbrough Tower Online Submission Notes:• Please follow carefully the instructions given on the Assignment Specification• When Extenuating Circumstances (e.g. extension) has been granted, a fullycompleted and signed Extenuating Circumstances form must be submitted to theSchool Reception or emailed to scdt-assessments@tees.ac.uk. Central Assignments Office (Middlesbrough Tower M2.08) Notes:• All work (including DVDs etc) needs to be secured in a plastic envelope or a folderand clearly marked with the student name, number and module title.• An Assignment Front Sheet should be fully completed before the work is submitted.• When Extenuating Circumstances (e.g. extension) has been granted, a fullycompleted and signed Extenuating Circumstances form must be submitted to theSchool Reception or emailed to scdt-assessments@tees.ac.uk. FULL DETAILS OF THE ASSIGNMENT ARE ATTACHEDINCLUDING MARKING & GRADING CRITERIA ICA-CIS4035-N-CT20-ADSMachine LearningCIS4035-NIn-course AssessmentOverview of RequirementsAssessment for Machine Learning (CIS4035-N) requires you to develop machine learning applicationsand make predictions about unseen data. The summative assessment for this module is via in-courseassessment (100%) which will evaluate all learning outcomes (see below).The assessment will emulate the “shared task” framework appearing in several machine learningvenues, in which participants are supplied with a task description and annotated data and mustdevelop a machine learning solution that makes predictions about an unannotated data set.The assessment for this module is individual and it contains two elements:1. The first element (50%) consists of developing a Machine learning application and itspredictions on the labelled or unlabelled dataset (50%) to assess Learning Outcomes 2, 3, 4, 5and 6 [50 points].2. The second element (50%) consists of a conference-style paper of approximately 2,000 wordsthat reports on the development of their system (50%) and it will assess Learning Outcomes1, 2 and 7 [50 points].The purpose of this assessment is to is to demonstrate achievement of the module learning outcomes(see Learning Outcome section).Further details are given below and there will be a supporting briefing session on the ICA.Submission of materials must be made via Backboard to the link provided. The submission date isspecified in the submission schedule.ICA-CIS4035-N-CT20-ADSLearning OutcomesPersonal and Transferable Skills1. Select, apply and defend the selection and application of machine learning methodologies andexperiments in academic reports.2. Demonstrate a systematic understanding of machine learning algorithms and their selection forsolving a specific problem.Research, Knowledge and Cognitive skills3. Investigate state-of-the-art machine learning algorithms.4. Design appropriate representations of machine learning problems for input into machine learningpackages and critically evaluate their effectiveness.5. Design and evaluate neural network configurations and learning mechanisms for sampleproblems.6. Analyse empirical results of the selected machine learning algorithms and justify the performance.Professional skills7. Autonomously implement and evaluate appropriate machine learning technique for particularlearning tasks, taking into consideration professional, ethical and legal issues.Task DescriptionProblems in machine learning vary from domain to another. In this coursework, you will select adataset related to a real-world problem that best suits your area of interest. There are abundant ofwebsites that provide publicly available datasets. A categorised list of datasets from GitHub can befound at https://github.com/caesar0301/awesome-public-datasets. The UCI Machine LearningRepository at https://archive.ics.uci.edu/ml/index.php is another long-standing source of benchmarkdatasets for data mining and machine learning research. Kaggle (https://www.kaggle.com/datasets)has interesting real-world problems and datasets.You can select a dataset from the above sources, or another one that is available online. The datasetshould be publicly available. The chosen dataset should have a minimum of 1,000 instances (rows)ICA-CIS4035-N-CT20-ADSand a minimum of 5 attributes (columns). You have to complete the following stages in thisassignment:1. Define the problem for the selected data set and identify the machine learning algorithms thatare applicable to this problem.2. Data exploration and preparation: The nature of the dataset may dictate some dataexploration and preparation that can help inform the solutions. For example, higherdimensional datasets (those with too many attributes/columns) may require applying a datareduction method like Principal Component Analysis (PCA).3. Propose solutions: In this step, you will propose three machine learning algorithms that areapplicable to the selected data set/problem.4. Design, implementation, modelling and evaluation: design, model and implement theproposed solutions and critically evaluate the solutions. Use appropriate visualisation for theresults.5. Reflect on professional, ethical, and legal issues in relation to the problem and the data set.Element 1 Deliverable – Contribute 50% of the Module MarkElement 1 will assess learning outcomes LO 2, 3, 4, 5 and 6.Deadline: 14/05/2021What to Hand-InSubmission method is online on Blackboard. You are required to submit a file in a pdf format viaBlackboard that includes all source code and screenshots from your experiments appropriatelylabelled and commented.You are required to submit copy of the source code and screenshots from your experimentsappropriately labelled and commented via Blackboard.The code and experiments will be assessed on:• Appropriateness of machine learning algorithm selected for the given task.• Quality of software architecture and implementation.• Quantitative performance of application.ICA-CIS4035-N-CT20-ADSElement 2 – Contribute 50% of the Module MarkElement 2 will assess learning outcomes LO 1, 2 and 7.Deadline: 14/05/2021What to Hand-In• A case study report maximum of 2000 words that documents the process of the entire casestudy, including data set, problem, data preparation and exploration, selected algorithms,critical evaluation and justification of the algorithms and findings.• Submission method is online on Blackboard. You are required to submit a file in a pdf formatvia Turnitin on BlackboardThe hand-in is electronically via Blackboard, all deliverables shall be labelled with project name, yourstudent name and university number.The report will be assessed on:• understanding of machine learning task• review of relevant literature• development methodology• justification of design decisions• consideration of professional, ethical, and legal issuesThe report could broadly include the following sections:• Abstract• Introduction (introduce the problem and its significance, write short literature review ofrelated work)• Data exploration and features selection• Experiments• Results• Discussion, Conclusions and Future Work• ReferencesThese are generic section titles, which you may adapt appropriately to the application/problem thatis investigated. You may include sections describing modifications of algorithms or developments thatare novel and specific to your work.ICA-CIS4035-N-CT20-ADSOutline Marking SchemeYour submission will be assessed according to the following criteria:1. Machine Learning application [50 points].2. Report (conference-style paper) [50 points].Below is a provisional indication of the criteria applied to determine points for each element.Please note:Exceptionally, whilst points are allocated to specific parts, outstanding work in one area may be usedto trade-off points against poorer work in another area. Machine Learningapplication [50 points]Source Code Documentation and DemoExcellent70% and aboveClear evidence of running the experiments with code that is excellentlyorganised and commented.Machine learning algorithms selected are appropriate for the given task.Excellent quality of software architecture and implementation.Excellent quantitative performance of application.Deep understanding shown.Very Good60% – 69%Very good evidence of running the experiments with code that is wellorganised and commented.Machine learning algorithms selected are appropriate for the given task.Very good quality of software architecture and implementation.Very good quantitative performance of application.Very good understanding.Satisfactory50% – 59%Satisfactory evidence of running the experiments with code that isorganised and commented.Machine learning algorithms selected are appropriate for the given task.Satisfactory quality of software architecture and implementation.Satisfactory quantitative performance of application.Satisfactory understanding.FailLess than 50%Little evidence of running the experiments with code that is not wellorganised and commented.Machine learning algorithms selected are not appropriate for the giventask.Poor quality of software architecture and implementation.Poor quantitative performance of application.Poor understanding. ICA-CIS4035-N-CT20-ADS NSNON-SUBMISSIONN/A Report[50 points]Academic Quality of the PaperExcellent70% and aboveExcellent technical quality (rigour of the experiments, data preparation,justification and correct application of the selected algorithms andsuitability of the selection).Produced and demonstrated a comprehensive, high quality solution tothe problem. Sufficient information for the reader is provided toreproduce the results.Outstanding evidence of systematic review using multiple high qualityacademic sources. Logical, clear development of narrative. High qualityreferences and citations.Outstanding evaluation and discussion of the significance of the results(Why the results are important? How does the paper advance the stateof the art? How would the results be useful to other researchers orpractioners? Is this a “real” problem or a small “toy” problem?).Legal, social, ethical, security and professional issues fully considered.A paper, which could be, with minor modifications, suitable for apublication – or form the basis for a postgraduate project. There is someelement of a novel approach to the problem or novel use of techniques.Very Good60% – 69%Very good technical quality.Produced and demonstrated very good quality solution to the problem.Sufficient information for the reader is provided to reproduce the results.Very good evidence of systematic review using multiple high qualityacademic sources. Logical, clear development of narrative. Appropriatereferences and citations.Very good evaluation and discussion of the significance of the results.Legal, social, ethical, security and professional issues fully considered.Satisfactory50% – 59%Satisfactory technical quality.Produced and demonstrated good quality solution to the problem.Good evidence of reviewing multiple academic sources. Some referencesand citations.Good evaluation and discussion of the significance of the results.Legal, social, ethical, security and professional issues fully considered.FailBelow 50%Not adequate technical quality.Produced and demonstrated a solution to the problem, which is flawed,despite some effort.Poor evidence of reviewing academic sources. ICA-CIS4035-N-CT20-ADS Little evaluation and discussion of the results.Little consideration of legal, social, ethical, security and professionalissues.Narrative difficult to follow. Poor quality of references and citations.NSNON-SUBMISSIONN/A Deliverables & SubmissionYou are required to submit your work to Blackboard via the assessments link by the due date.You may use a zip file to package your submission artefacts (i.e., the fprg files and yourreflective report). All the submitted files should be labelled as follows for identificationpurposes:studentID_lastname_firstname.zip (e.g. x1234567_smith_jane.zip)Your reflective report should also be labelled in a similar manner with your student ID.Make sure your student ID and name is present on all documentation you submit.LogisticsAfter the ICA briefing has been given, you will be provided with opportunities to progressyour in-course work during some timetabled sessions. Feedback – but not points – will begiven on your work in progress to assist you in submitting a considered and well-developedICA submission.Academic Misconduct and PlagiarismPlease note that the University takes the issue of academic misconduct and plagiarismvery seriously. You should not copy anyone else’s work or use copyright materialswithout due acknowledgement.
