Informatica Economică vol. 19, no. 3/2015 89DOI: 10.12948/issn14531305/19.3.2015.08A Quantitative Approach to Information Systems Audit in Small andMedium EnterprisesUma VIJAYAKUMAR1, D. ILANGOVAN21Research & Development Centre, Bharathiar University, Coimbatore, India.2Dept. of Commerce, Annamalai University, Annamalainagar, India.bv_uma@yahoo.com, dil2691@yahoo.co.inAn Information Systems (IS) Auditor performs several audit related functions in a Small andMedium Enterprise (SME) such as preparation of a written IS audit procedure, comparison ofactual IS configuration with documented configuration standards, assess whether IS assetsare secure, check the access rights for users and system services, check for the presence of ISsecurity procedures and finally analyze transactions in an information system. The currentwork focuses on a quantitative approach to measure the effectiveness of the IS audit functionsin selected small and medium enterprises. The variations in KPI scores between sectors andregions are analyzed for the sample SMEs. Finally, the operational best practices for ISAuditors working in SMEs are suggested.Keywords: Information Systems (IS), IS Audit, Key Performance Indicator (KPI), Small andMedium Enterprise (SME), Maturity Level IndexIntroductionAn enterprise is mainly involved ineconomic activities. It can be categorized asLarge, Medium or Small depending on thelimits for investment, number of employees,balance sheet and total turnover. SMEs arecontributing for economic developmentacross the world. Information Systems Auditplays an important role in SMEs for runningcomputer based application systems.Information Systems Audit ensuresprotection of IS assets and maintains dataintegrity. It also helps in achievingorganizational goals and facilitates efficientusage of resources [1]. SMEs in the modernenvironment extensively make use ofinformation system resources. This willensure smooth flow of information betweenvarious sub systems and improves thebusiness processes as well. An InformationSystems (IS) Auditor performs several auditrelated functions in a Small and MediumEnterprise (SME) such as preparation of awritten IS audit procedure, comparison ofactual IS configuration with documentedconfiguration standards, assess whether ISassets are secure, check the access rights forusers and system services, check for thepresence of IS security procedures andfinally analyze transactions in an informationsystem.2 ObjectivesThe objectives of the present work can bestated as follows:1) To assess the existence of IS Auditexpertise in SMEs with reference tothe KPI- Maturity level Index.2) To study the variations in the KPIscores between the sectors andregions.3) Suggest operational best practices forIS Auditors with respect toInformation Systems Audit in SMEs.3 Related WorkThe article by Tommie W. Singleton [2]analyses the four phases of the ControlsDevelopment Life Cycle, viz., design,implementation, operational effectivenessand monitoring. The design phase involvesIS controls pertaining to Top Management,Quality Assurance Management, OperationsManagement, Security Management,Systems Development Management, DataResources Management, ProgrammingManagement and User ApplicationsManagement. The implementation phaseshould carry out the controls listed in the190 Informatica Economică vol. 19, no. 3/2015DOI: 10.12948/issn14531305/19.3.2015.08design phase. The operational effectivenessphase is concerned with ability of thecontrols to perform their goals (e.g. prevent amaterial misstatement). The monitoringphase involves continuous auditing on thecontrols and proper review of the changemanagement procedures.The monograph by Khabib [3] gives anoverview of controls for applications, datacentre operations and access security. It alsogives an overview of computer based audittechniques to independently test computerdata. Jim Kaplan proposed [4] a simplifiedrepresentation of the enterprise informationenvironment. He gave an overview of ISaudit process, accuracy, consistency andreliability of data, controls for the coreprocesses and application systems.The fourth annual Information Systems AuditBenchmarking Survey conducted byInformation Systems Audit and ControlAssociation (ISACA and Protiviti in 2014 [5]highlights the challenges and concernsrelating to computer and internet security, ISstaffing and resources, IS risk assessment andIS audit reporting structure.4 Present Scenario of Information SystemsDeployment in SMEsSMEs in the modern context are making useof IS infrastructure in a big way in theirnormal operations. However, the IS Audit isyet to evolve significantly in many SMEs.The internet based applications face a lot ofproblems related to information security inSMEs. The fraudulent websites createproblems for SMEs by stealing personal andconfidential data such as password, creditcard number and so on. The Federal TradeCommission has stated that the number ofphishing attacks have increased to a largeextent during the last five years [6]. Thephishing sites target individuals, banks,SMEs, e-commerce websites and governmentorganizations. When the recipient has keyedin his/her personal details, the cybercriminals gain access to the recipient’sconfidential details and cause problemsrelating to the recipient’smoney/credit/account. SMEs also face threatsfrom external as well as internal sources. Forexample, computer data are stolen usingmalwares like Trojans / viruses. ComputerCrime as defined by the Association ofInformation Technology Professionals(AITP) include unauthorized actionsinvolving usage, access, modification anddestruction of hardware, software, data ornetwork resources, release of information,copying of software tools, causing denial ofservice attack to genuine users and usingcomputer & network resources to illegallyobtain information [7].SMEs face external threats from Trojans,Spyware, Viruses and Worms for their ISinfrastructure. These threats penetrate intoweb browsers, desktop computers and e-mailservers. The common assumption that smallbusinesses are too small to be targeted bycomputer threats is not true in the presentscenario [8]. This is the background againstwhich the current work will investigate theobjectives listed earlier.5 Research MethodologyThis section deals with Data Collection,Sources of Data, Period of the study,Geographical area for the study and theSampling Frame.Data Collection: The primary data werecollected from the sample respondentschosen from the Stakeholders / IS Auditors inselect SMEs in India and the UAE. AQuestionnaire has been prepared toadminister upon them for collection offirsthand information from samplepopulation. The strategies for evidencecollection and evaluation include thefollowing: discussion, observation, webbased survey using Google Documents andtelephonic interview.Sources of Data: The sources of data aboutSMEs have been taken from Annual Reportsof Ministry of MSME, Govt. of India,Annexure-XII and Mohammed Bin RashidEstablishment for SME Development, Dubai,UAE, for the period 2009-2010.Period of the study: The period of the studyfor making worthwhile analysis has beenchosen as 2010-2011.Informatica Economică vol. 19, no. 3/2015 91DOI: 10.12948/issn14531305/19.3.2015.08Geographical area for the study: Thegeographical area for the study includes twocountries, namely India and the UAE bytaking into consideration the feasibility andaccessibility factors. The four regionsconsidered in India include: North, South,East and West. The four regions consideredin the UAE include: Abu Dhabi, Dubai,Sharjah and Other Emirates.Sampling FrameSTRATIFIED SAMPLING method has beendeployed in the current work. The Strataconsidered for the study comprises of threesectors Manufacturing, Services and Trading.Equal sample selection from each stratumhas been considered. Although, the stratasizes are different, it is required to comparethe differences among the strata [9]. Thesample size has been chosen using standardtable for a given set of criteria [10]. The setof criteria considered in the present work are:Confidence level=95% and level ofprecision=5%.The sample SMEs chosen in each sector hasbeen 4. The Total number of Stakeholders /Information Systems Auditors selected fromboth countries (India and the UAE) has been96. (i.e. 2 countries * 4 regions * 3 sectors *4 SMEs/sector * 1 Stakeholder/IS Auditorper SME = 96).Framework of AnalysisThe statistical measures include thefollowing: Measurement Scale, Mean andTwo-way ANOVA. The KPI (KeyPerformance Indicator) considered for thestakeholder / IS Auditor has been MaturityLevel Index.Testing of HypothesisHypothesisSectoral and Regional Variations (Twoway ANOVA) within a countryThe regional and sectoral KPI scores within acountry do not vary or the differencesbetween them are not significant. This isexpressed as under:H0 : (µs1)KPI = (µs2)KPI = (µ3)KPIH0 : (µr1)KPI = (µr2)KPI = (µr3)KPI= (µr4)KPIH1 : (µs1)KPI ≠ (µs2)KPI ≠ (µs3)KPIH1 : (µr1)KPI ≠ (µr2)KPI ≠ (µr3)KPI ≠ (µr4)KPIµs1-s3 = Sectoral Mean Score of each of the 3sectors.µr1 – µr4 = Regional Mean score of each ofthe 4 regions.Hi : There is no significant interactionbetween the two factors sector and region.The alternate hypothesis states that thereexists an interaction between the two factorssector and region. The above hypothesis istested with the primary data pertaining to theKPI: Maturity Level Index.Measurement ScaleQuestions involving (Yes/No) responses arerepresented in a 3 point scale as, shownbelow:Yes No Not Applicable+1 -1 0 is left for dummyresponse treated as neutral.Scaling technique helps in transformingqualitative aspects into quantitativeconstructs. The response scores are finallyconverted into a cumulative score, which isthen represented on a ten point scale. This isdone by using the formula as stated below:Appendix F6 QuestionnaireThe following questionnaire has beenadministered to the stakeholders / IS auditorsin the sample SMEs using a web basedsurvey.1. Do you have a written IS audit /independent review program?2. Do you have internal IS auditor in yourorganization?3. Does IS audit coverage include acomparison of actual system configurationsMeasurement Scale(index) =(Cumulative Score)Actual(Cumulative Score)Maximum x 10(1 = Low,10 = High)92 Informatica Economică vol. 19, no. 3/2015DOI: 10.12948/issn14531305/19.3.2015.08to documented/baseline configurationstandards?4. Does IS audit coverage include assessingcompliance with information securityprogram requirements?5. Does IS audit coverage include assessingusers and system services access rights?6. Is Information Systems Audit involved inyour risk assessment process?7. Do you use any IS Audit Software Tool? IfYes, specify the name.8. Do you have documented IS Securityprocedures?9. Do you have a concurrent IS audit(embedding audit modules in an applicationsystem to provide continuous monitoring of asystem’s transactions) mechanism for all ofyou?7 Quantitative AnalysisThis section deals with quantitative analysisinvolving the KPI – Maturity Level Index.The Maturity Level Index indicates theeffectiveness of the following IS functions ina SME:i) Monitoring IS operations.ii) Compliance with IS practices.The current work is aimed at studying thematurity level of IS Audit in sample SMEs.The observed values for the KPI – MaturityLevel Index are represented on a scale of 10.The Sectoral and Regional Variations in boththe countries are analyzed using Two-wayANOVA. Finally, the hypothesis is testedand appropriate inferences are made for theKPI – Maturity Level Index. The results ofthe above hypothesis testing with the primarydata for India and the UAE are shown belowin TABLEs 1.A and 1.B.ObservationIn case of India and the UAE, the averagevalues for Maturity Level Index observed for agiven pair of sector and region are shown inTABLEs 1.A and 1.B. Their values varybetween 2.778 to 5.8333.FindingsIn the case of India, the sectoral and regionalvariations are summarized as follows.Between Sectors: The calculated value for F(0.55686) is less than the table value F crit(3.25945) at 5 Percent level of significance.Hence, the null hypothesis is accepted.Between Regions: The calculated value for F(0.4569) is less than the table value F crit(2.86627) at 5 Percent level of significance.Hence, the null hypothesis is accepted.In case of the UAE, the sectoral and regionalvariations are summarized as follows.Between Sectors: The calculated value for F(5.57561) is greater than the table value Fcrit. (3.25945) at 5 Percent level ofsignificance. Hence, the null hypothesis isrejected.Between Regions: The calculated value for F(2.16098) is less than the table value F crit(2.86627) at 5 Percent level of significance.Hence, the null hypothesis is accepted.InferenceThere are no significant variations in theMaturity Level Index scores, between thethree sectors in India. There are significantvariations in the Maturity Level Index scores,between the three sectors in the UAE. Thereare no significant variations in the MaturityLevel Index scores, between the four regions,in both the countries. There is no significantinteraction between the two factors sectorand region, in the determination of MaturityLevel Index scores, in both the countries.Table 1.A. Sectoral and Regional Variations in India – Maturity Level IndexSUMMARY East North South West TotalServices ManufacturingCount 4 4 4 4 16Sum 18.89 17.78 20.01 21.11 77.79Average 4.7225 4.445 5.0025 5.2775 4.86188Variance 3.61216 4.12923 3.71483 1.14056 2.62222 Informatica Economică vol. 19, no. 3/2015 93DOI: 10.12948/issn14531305/19.3.2015.08Count 4 4 4 4 16Sum 17.78 20 22.22 21.12 81.12Average 4.445 5 5.555 5.28 5.07Variance 2.47903 5.36133 2.47903 0.3136 2.30656 TradingTotalSource ofVariation SS Df MS F P-value F critSample 3.24815 2 1.62408 0.55686 0.57786 3.25945Columns 3.99764 3 1.33255 0.4569 0.71407 2.86627Interaction 2.10395 6 0.35066 0.12023 0.99327 2.36375Within 104.993 36 2.91648Total 114.343 47 Count 4 4 4 4 16Sum 15.55 18.89 18.89 17.78 71.11Average 3.8875 4.7225 4.7225 4.445 4.44438Variance 2.06463 3.61216 3.61216 2.47903 2.47755Count 12 12 12 12Sum 52.22 56.67 61.12 60.01Average 4.35167 4.7225 5.09333 5.00083Variance 2.35583 3.62948 2.80488 1.24121ANOVA – Two-Factor With ReplicationSource: Primary Data 2010-2011Table 1.B. Sectoral and Regional Variations in the UAE – Maturity Level Index SUMMARY AbuDhabi Dubai Other Emirates Sharjah TotalManufacturingCount 4 4 4 4 16Sum 20 21.1111 21.1111 18.8889 81.1111Average 5 5.27778 5.27778 4.72222 5.06944Variance 2.05761 1.13169 6.893 1.13169 2.29938ServicesCount 4 4 4 4 16Sum 21.1111 23.3333 17.7778 20 82.2222Average 5.27778 5.83333 4.44444 5 5.13889Variance 1.13169 2.77778 2.46914 0.41152 1.62551TradingCount 4 4 4 4 16Sum 12.2222 21.1111 15.5556 11.1111 60Average 3.05556 5.27778 3.88889 2.77778 3.75Variance 1.13169 1.13169 0.41152 0.41152 1.62551TotalCount 12 12 12 12 94 Informatica Economică vol. 19, no. 3/2015DOI: 10.12948/issn14531305/19.3.2015.08 Sum 53.3333 65.5556 54.4444 50Average 4.44444 5.46296 4.53704 4.16667Variance 2.24467 1.44968 3.02095 1.59933ANOVA – Two-Factor With ReplicationSource ofVariation SS Df MS F P-value F critSample 19.5988 2 9.79938 5.57561 0.00777 3.25945Columns 11.394 3 3.79801 2.16098 0.1096 2.86627Interaction 8.59053 6 1.43176 0.81463 0.5657 2.36375Within 63.2716 36 1.75754Total 102.855 47 Source: Primary Data 2010-20118 Operational Best Practices for ISAuditors Working in SMEsIS Auditors can conduct audit for automatedand semi-automated systems in SMEs andensure compliance with established IS auditprocedures. It is necessary that the risksrelating to software applications and ITinfrastructure (like security threats andviruses) are analyzed and appropriaterecommendations are made to mitigate risks.IS auditor should examine the IS policies andstandards followed in a SME.Operating systems such asWindows/Linux/Mac should be well updatedwith the latest and necessary patches. Therelease of such systems should be audited ata regular basis so there are no securityloopholes in them which might lead to an OS(operating system) level attack.In-house / External applications such as ERP,HR, and Payroll etc should also be auditedtowards their functionality. There should beproper controls for the Information Systemswhich are purchased from third partyvendors.An IS auditor should examine the security ofinformation systems in a SME, as describedbelow:Logical access1. Passwords must be set according to thestandards set by the IS Securitydepartment.2. The passwords set for the last 5 timesshould not be the same.3. There should be a certain time interval(say 30 to 90 days) during which thepasswords must be changed.Physical access1. No outsider should be able to enter thecompany’s premises until and unless theentry procedure is followed. Each andevery outsider should deposit an identity.2. Laptops must be locked to theworkstations in case the owner of thelaptop has gone outside.3. The access to the server room shouldalways be recorded and should beaccessed only by the authorizedpersonnel.4. Server room should be protected fromnatural disasters.Backup PolicyAn IS auditor should examine the backuppolicy in a SME, as described below:1. Frequent backup (daily, weekly, monthlybasis) of the data within the systems /applications should be taken and kept in asecure place in the premises of SME.2. For critical data, the backup can be storedadditionally at a secure remotesite/location.Disaster Recovery Plan (DRP) / BusinessContinuity Plan (BCP)An IS auditor should examine the DRP/BCPin a SME, as described below:1. Availability of DRP/BCP Plan.2. Frequency of DRP/BCP drill.3. Results of the drill must bemaintained for future reference.Informatica Economică vol. 19, no. 3/2015 95DOI: 10.12948/issn14531305/19.3.2015.089 ConclusionThis paper dealt with a quantitative approachto assess the maturity level for IS audit insample SMEs.. The variations in KPI scoresbetween sectors and regions have beenanalyzed for the sample SMEs. Theoperational best practices for IS Auditorsworking in SMEs have been suggested.References[1] R. Weber, Information Systems Controland Audit, Pearson Education, 2008.[2] T. W.Singleton, “What Every IT Auditorshould know about controls: The ControlsDevelopment Life Cycle”, ISACAJournal Vol 3, 2009.[3] Khabib, Information Technology Audit,General Principles, IT Audit MonographSeries, available athttp;//khabib.staff.ugm.ac.id, 2010, pp. 1-25.[4] J. Kaplan, Information Integrity Auditnet:Monograph Series, available atwww.infogix.com, 2003, pp.4-24.[5] ISACA and Protiviti, A Global Look atIT Audit Best Practices, 2014, availableat www.isaca.org.[6] Neesa Moodle & Issacs, , Banks offernew software to keep phishers out of youraccount, June 5, 2010, available atwww.mi2g.com.[7] J. A O’Brien & George M Marakas,Management Information Systems, 7thedition, Tata Mcgrawhill Pub., pp. 429-430, 513-515, 2009.[8] Snap Gear Secure ComputingCorporation, Security Solutions for SmallBusinesses and Remote Branch Offices,2009.[9] C.R. Kothari, “Research Methodology:Methods and Techniques”, New AgeInternational (P) Ltd., 2007, pp.63.[10] D I Glenn, “Determining Sample Size”,University of Florida IFAS Extension,April 2009, pp.9-11, available from:http://edis.ifas.ufl.edu/pd006.Uma VIJAYAKUMAR is a Chartered Accountant, with specialization inInformation Systems Control and Audit. She is a research scholar in the dept.of Commerce at Bharathiar University, Coimbatore, India. She is a FellowMember of Institute of Chartered Accountants of India. She has 25 years ofexperience in finance and auditing.Dr. D ILANGOVAN is a Professor in the Department of Commerce,Annamalai University, India, with specialization in Cooperation & IndustrialMarketing. He has over 25 years of experience in teaching and research. Hehas several publications to his credit at international and national level.
