Matters Arising from our Information System Audits Overview Introduction As part of our audit of the Province, we document and test controls in significant information systems. Each year, we test controls and make recommendations if we believe controls should be improved. This chapter reports our findings on those systems where we recommended changes to departments. In this chapter This chapter contains the results of the following audits: Audit See section Department of Social Development – NBFamilies A Department of Social Development – NBCase expenditures B Section A Department of Social Development NBFamilies Overview Introduction The purpose of this section is to discuss our findings and to make recommendations from our audit of the NBFamilies system. We conducted this work in the Winter and Summer of 2008. In this section This section contains the following topics: Part Topic Page A Background, Objective and Scope 141 B General Computer Controls 146 C Results from Sample Testing 162 D Accounting Issues 176 Part A Background, Objective and Scope Overview Introduction The purpose of this part is to provide background information for the NBFamilies system and to explain the objective and scope of our work. Contents This part contains the following topics: Topic See Page Background 142 Objective and Scope 144 Background NBFamilies history The following points provide a history of the development of the NBFamilies system. * The NBFamilies system was originally part of the Client Service Delivery System (CSDS) in the Department of Health. * In April 2000, the government of the day reorganized the Department of Health and Community Services and the Department of Human Resources Development, creating a new department called Family and Community Services (FCS). * The Family and Community Support Services section (FCSS) of the Department of Health and Community Services was transferred to this new department. * Staff of the FCSS section needed access to the CSDS system. Consequently, FCS made a copy of the CSDS system, renamed the system NBFamilies and continued system development. (The functionality needed by the FCSS staff had not yet been developed.) * FCS completed the NBFamilies system in November 2004 and rolled out the system over the next seven months. * By July 2005, NBFamilies was fully implemented in the Department. * In 2008, the Department’s name was changed to Social Development (SD). Programs that use NBFamilies Various departmental programs are paid using NBFamilies. The total amount of payments processed by the system is approximately $251 million for the fiscal year ended 31 March 2008. These payments relate to various departmental programs as outlined in the table below. Types of Services Amount ($ million) Child Protection $ 14.7 Child In-care Services 22.1 Long Term Care Services 183.9 Various Other Services 30.3 Total $ 251.0 Continued on next page Background, Continued Management and support The NBFamilies system was developed by consultants who were under contract with the Department. The Department recently signed another agreement with these consultants for the continued support of the NBFamilies system. The consultants are responsible for supporting the application and maintaining the software. The Department also has a contract with Aliant for services. Aliant is responsible for monitoring, housing, backup, patching and upgrading the Unix operating system. Within the Department a number of groups and/or individuals are responsible for managing various aspects of NBFamilies. They are: Groups/Individuals Role with NBFamiles Business Owner Ensures the functionality of the system meets users’ needs System Owner Responsible for the technical architecture and technical direction Business Support Team Responsible for providing support to end users of the NBFamilies Information Technology Director Responsible for oversight for the Department and contract management. All of these individuals meet regularly to manage and support the system. Minutes from the various meetings are recorded and kept on file. Objective and Scope Objective The objective of our audit was to obtain audit evidence on the completeness, accuracy, valuation and existence of payments made by the NBFamilies system to support our opinion on the Province’s financial statements. Basic audit approach Because of a change in generally accepted auditing standards, we are now required to document internal controls for all major systems in the Province. The auditor is required to evaluate internal controls, but not necessarily to rely upon them. If we determine internal controls are in place, and that they appear to be operating throughout the period under examination, we can test them and rely on them as audit evidence to support our audit opinion. We can also obtain sufficient and appropriate audit evidence by testing transactions and not relying on internal controls. Typically, once internal controls are documented, it is more efficient to test and rely on internal controls rather than perform detailed transaction testing. Our planned reliance on controls Initially, we planned to rely on the internal controls for our audit work on payments processed by the NBFamilies system, provided our initial evaluation of controls indicated that we could rely on them. However, because of a number of challenges encountered throughout this audit, we were unable to do this. As a result, all of our audit evidence was obtained from tests of transactions and we placed no reliance on internal controls. Audit challenges Through out this audit, we encountered numerous challenges that prevented us from using the more efficient controls approach. This increased the amount of time needed to complete our work. Some of these challenges were: * Delays in providing documentation. Our auditors requested information at the start of our audit. Some key information needed for testing controls, took two to three months for us to receive. Information was provided to us on the last day of our audit. Also, information was not provided in a format suitable to our auditors. * Regional nature of service delivery. This Department is one of the largest in government with numerous regions throughout the Province. Obtaining information from and having discussions with the regions is time consuming and difficult. Continued on next page Objective and Scope, Continued Audit challenges – continued * Complexity of the range of the programs supported by the system. The NBFamilies system processes payments for various programs in the Department. Finding the person responsible for a particular area is a challenge. Findings from evaluation of controls Although we were unable to complete our testing of internal controls because of the various challenges we have outlined, we do have a number of observations and recommendations that we are reporting. These are discussed in the section General Computer Controls starting on page 146. Results of detailed testing Our detailed tests of transactions related to the following audit areas. Audit Area Number of Items Adult residential facilities 13 In-home support services 9 Children under guardianship 2 Child protection 1 Subsidized adoption 1 Alternative family living 1 Community based services 1 Total 28 Part B General Computer Controls Overview Introduction This part discusses our findings relating to general computer controls. Contents This part contains the following topics: Topic See Page NBFamilies Security 147 Granting System Access 151 Terminating Inactive Accounts 153 Program Changes – Plan Approval Process 154 Approval of Design Specifications 155 Program Changes – Testing Signoff 156 Disaster Recovery and Business Continuity Planning 158 Operational Framework 160 Work of Audit Services 161 NBFamilies Security Objective To ensure that the NBFamilies system has adequate security to protect its information. Background Complex computer applications typically have numerous points where confidential data can be accessed. Each of these access points needs to have an appropriate level of security to protect the accuracy, integrity and confidentiality of the data. From our limited review of the NBFamilies system, we have identified three key data access points - the application production environment, the database environment and the Unix operating system environment. We asked the Department to complete a basic security questionnaire for each of these access points. We made a number of observations and recommendations based on the answers to these questionnaires. Application production environment The application production environment is how users of the system login to the application. In the case of NBFamilies, approximately 1,150 users have access to the system using a user-id and password. We noted a number of good security practices in place for this environment. For example: * password composition is strong, * passwords are changed regularly, * each user is uniquely identified by the system, and * the system automatically terminates the session after a period of inactivity. We also noted one area where the Department should improve security by monitoring user access logs. Logs The Department maintains a log that records logins, logoffs, locked accounts from failed login attempts and other login account usage. No one in the Department, however, has been assigned to review this log for unusual activities. Reviewing access logs is important to ensure unauthorized access attempts are detected. For example, if an unauthorized user was trying to gain access to the system, this activity would be recorded in the log. The Department, however, would not be aware of this because it is not reviewing the logs on a regular basis. Continued on next page NBFamilies Security, Continued Database environment The production database stores all of the data for the NBFamilies application. The accuracy and integrity of the data could be easily damaged if access to the database is not strictly controlled. The production database has five accounts. Consultants are responsible for the administration of the database and for changing the passwords on these accounts. From our discussions with the Department, we were informed that the passwords are not changed regularly and that changing these passwords could cause major issues with the application. We were told that the passwords for two of the five database accounts were changed last year when the consultants staff changed. However, the passwords for one of the remaining three accounts have not changed for at least three years. Best practices advocate changing database passwords on a regular basis. Not changing the database passwords on a regular basis increases the risk of unauthorized access to the data. We have concerns about an application system where passwords cannot be easily changed when required, especially when that application contains confidential information about New Brunswickers. Continued on next page NBFamilies Security, Continued Unix operating system environment Unix is the operating system used to run the NBFamilies application. The operating system is managed by Aliant, a third party service provider. Aliant is responsible for monitoring, housing, patching, and upgrading the operating system. Application users do not login to the operating system to access the NBFamilies application; however, nine system support users have accounts to access the operating system. Typically, system support users have access to all the files stored on the server, including the application database. While the support users could not directly login to the production database, they would have the ability to delete the database. Thus, these users are extremely powerful and strong security should surround their system accounts. While we did not conduct an in-depth review of the Unix operating system, we noted a number of practices that are not normally associated with “good” security procedures. Password composition Password composition does not conform with PNB standards. The PNB password standards which were effective as of March 2003 require baseline security for user accounts. The standards indicate that administrator accounts are a special class of user accounts for which additional baseline security requirements may apply. One of the requirements of the standard relates to password composition. The standard requires passwords to contain a combination of upper case, lower case, numeric characters and special characters. The passwords for the Unix operating system users do not meet these requirements. We believe given the powerful level of access of the Unix users, the password requirements should meet or exceed the requirements of the PNB standards. Strong password composition reduces the risk of easy-to-guess passwords thus reducing the risk of unauthorized system access. Password life Users are not required to change passwords. The PNB standards also require users to change passwords at least every 60 days. The settings for the Unix operating system do not require passwords to be changed regularly. Not changing passwords on a regular basis increases the risk of unauthorized system access. Continued on next page NBFamilies Security, Continued Threat risk assessment Throughout this section, we have highlighted a number of areas where we believe security processes should be improved. As we mentioned in the background of this section, our limited review has identified three access points where data could be comprised. Additional access points could exist where the data could be compromised. One manner in which the Department can determine where the security weaknesses exist and the potential for data exposure is through a Threat Risk Assessment (TRA). A threat risk assessment will identify potential threats, the likelihood of their occurrence and the measures that should be in place to protect the data. The Department completed a TRA for one aspect of the system, the Vendor Electronic Invoicing System (VEIS). As well, the Department had a third party vendor perform a security assessment for the VEIS. We would like to see the Department complete a TRA for the entire NBFamilies system to identify potential threats and risk of exposure. We recommended We recommended the Department monitor failed login attempts to the production database. We recommended the Department develop a process whereby database passwords can be changed regularly. We recommended the Department improve Unix security so that passwords are changed regularly and that password composition complies with the PNB password standards. We recommended the Department complete a threat risk assessment for the NBFamilies system. Granting System Access Objective To ensure system access is approved by the business owner and that user access is not incompatible resulting in a lack of segregation of duties. Observations From our discussions with staff about the process for granting access to system users, we make the following observations. 1. The business owner does not approve user access requests. Business owners should approve access to ensure that: - the business owner approves who has access to sensitive confidential data, - the level of access is appropriate given the user’s job responsibilities, and - the profiles assigned to users do not conflict. 2. The Department should ensure that users are not assigned conflicting profiles. This is important so that segregation of duties is maintained. For example, a user should not be assigned a profile for creating and approving invoices. The business owner should be responsible for ensuring users’ profiles are not incompatible. We found cases where incompatible profiles were assigned to users. See example below. 3. The Department does not have a listing of incompatible profiles. As mentioned above, having this list is important so that approvers of system access are aware of incompatible profiles. Incompatible profiles From discussions with staff, we were told that the following profiles are incompatible: - NBFSW NBF Social Worker and NBFADM NBF Administrative Support From our review of the documents provided by the Department, we found three users with both of these profiles. The assigning of these conflicting profiles to users was an oversight and has since been corrected. Continued on next page Granting System Access, Continued Departmental comments From our discussions with the Department after our audit, we learned that its staff is in the process of looking into the area of user profiles and access and staff plan to improve this process. We recommended We recommended user access requests be approved by the business owner or approved delegate. We recommended a listing of incompatible profiles be prepared and provided to the individuals responsible for approving system access. We recommended users should not be assigned incompatible responsibilities. Terminating Inactive Accounts Objective To ensure inactive user accounts are terminated within 90 days as required by the PNB password standards. Findings The NBFamilies system does not automatically disable accounts after 90 days of inactivity. Terminating accounts after 90 days of inactivity is a requirement of the PNB password standards. We found 180 users had not accessed the NBFamilies system within the last 90 days. A breakdown of the 180 accounts is as follows. 63 users have never logged in 6 users had not logged in since 2006 86 users had not logged in since 2007 25 users had not logged in since February 2008 Promptly terminating user access reduces the risk of unauthorized access to confidential data. The Department should modify the system to disable inactive accounts or its staff should manually review and disable inactive accounts. Departmental comments The Department agreed with this comment. It indicated, however, in some cases accounts are needed for the hierarchical approval structure even though users do not need access to the system. Its staff will look into disabling these accounts because in addition to not complying with the PNB standards, having inactive user accounts ties up licenses and could increase departmental costs. We recommended We recommended the Department disable NBFamilies user accounts after 90 days of inactivity. Program Changes – Plan Approval Process Objective To ensure a process is in place to approve NBFamilies program changes. Scope There are three ways changes are made to the NBFamilies system. In this audit, we focused on changes made through the release process. Improvements needed in plan approval process The process for making program changes is described in detail in the Operational Framework. We were very impressed with this document and the process described within. Part of the release process is to create/approve a release plan. We found inconsistencies in the way the release plan was approved. For example, in release 11 the plan was approved in the minutes of the maintenance committee meetings. There was also an email from the business owner to one of the consultants indicating that the plan was approved. For release 12, there was only approval in the maintenance committee meeting minutes. Also, based on the length of time required by the Department to find documentation and from our review of documentation we received, we believe that finding the plan approval appeared to be a time consuming process. We expected to find a documented release plan, listing the changes to be included in the release with appropriate signatures from both the Department and the consultants. This release plan would be similar to the scope definition document prepared for other departmental systems. Instead, we found the approval process to be informal and inconsistent. Departmental comments The Department agreed with our comments and its staff have held meetings to discuss the best way to implement our recommendations. We recommended We recommended the release plans for the NBFamilies system be documented and approved by the consultants and appropriate members of the Department. Approval of Design Specifications Objective To ensure design specification documents are signed off before work is started. Background Design specification documents are prepared for most issues included in a release. These documents describe the nature of the change, the work to be done and how the change will be made to the application. Having a well documented design specification helps ensure the appropriate changes are made to the system in the most efficient and economical manner. Observations We found that design specifications were not approved by both the consultants and the Department before work begins on the related issue. We believe this sign off is important so that both the developer and the owner agree on the proposed method of changing the application. We noticed that the design specification documents have a front page where approval is required by both a consultant architect and a user authority. However, this page is not signed off. We noted from our work on other systems, design specification documents are signed off prior to work starting. We did find that sign-off is obtained once the work is complete. An “Approval/Sign off” sheet is attached to the design specification. This sheet is signed off by both the Department (after the release is put in production) and the consultants (before the release is put in production). Departmental comments The Department agreed with our comments and its staff have held meetings to discuss how to implement our recommendation. We were told that in some cases work on the changes happens at the same time as the design is developed. In these cases, once the design is finalized the document should be signed off. We recommended We recommended design specifications be signed off by both the consultant and the Department prior to work commencing. Program Changes – Testing Signoff Objective To ensure a formalized process is in place to test program changes prior to them being implemented into production. Background Testing of program changes is an integral step in the program change process. Changes that have not been properly tested could cause applications to produce incorrect results. Proper documentation and sign-off of changes is necessary to provide evidence that all changes were appropriately tested before the changes are put into production. Observations We reviewed the testing process for both release 11 and release 12. We saw evidence that the Department was testing program changes. However, we found inconsistencies in the sign off process to inform business owners that all changes were adequately tested and ready for production. For example: * For release 11: In the Weekly Issue Review Meeting minutes, we saw mention of the fact that all issues have been approved in UAT (User Acceptance Testing). We did not see any communication to the business and system owners that the testing was complete. * For release 12, we saw an email from the Business Support group to the consultants and the business owner stating that testing was complete and the tests were successful. The Department should develop a standard process for the sign-off of testing. This would include documented evidence that the testing was completed successfully and that the business owners are notified that testing is complete prior to sign-off to production. The business owners should be notified by the person responsible for ensuring that the testing is complete. Having a well defined process will help ensure untested changes are not put into production. Departmental comments The Department agreed with our recommendations and will develop a series of process-related improvements to address the issues identified. Continued on next page Program Changes – Testing Signoff, Continued We recommended We recommended the Department formalize the testing sign-off process for release changes. In particular, the Department should document and sign off that all testing is complete. We recommended the business owners be notified formally that the testing is complete before implementing the change into production. Disaster Recovery and Business Continuity Planning Objective To ensure the NBFamilies system is adequately protected from loss of data and is able to resume operations in the event of a disaster. Background Contingency planning and protection of data is an important aspect in the management of any mission critical system. Having documented and tested disaster recovery and business continuity plans helps ensure the protection of data and the continuation of service in the event of a disaster. Business Continuity Plan (BCP) A BCP is important as it identifies critical business processes and establishes the information and resources that are needed to ensure that these processes continue to operate in the event of a disruption in service. A BCP should incorporate components such as: * a business impact analysis, * human resource needs, * a backup and offsite storage program, and * emergency response procedures. Recovery Plan A subset of a business continuity plan is a recovery plan. This plan focuses on the recovery of the computer environment needed to support the critical business processes if a disruption should occur. A recovery plan should include components such as: * information technology processing resources; and * procedures, manuals and other hard copy documents that are required for the resumption of business applications. Observations – No disaster recovery plan The Department does not have a documented and tested disaster recovery plan or business continuity plan. Section 4.5 of the operational framework states: “The Department of Family and Community Services will need to complete a full Disaster Recovery Plan which covers all of their servers and environments including those used for NBFamilies.” The Department has yet to complete this plan. Continued on next page Disaster Recovery and Business Continuity Planning, Continued Observations – Recovery process not tested periodically One problem that we noted from our security questionnaire is that for each of the three environments (application, database and operating system), the Department does not test the recovery process periodically. Testing the recovery process is important to ensure data can be recovered and the system can be restored in the event of data loss. Observations – No business continuity plan The Department has not completed a business continuity plan for its key systems. We commented on the absence of a plan in our 2004 report on the NBCase system. However, a complete business continuity plan has still not been completed. We recommended We recommended the Department periodically test its recovery process for each of the three NBFamilies operating environment. We recommended the Department document and test a disaster recovery plan for the NBFamilies system. We recommended the Department complete a business continuity plan for its mission critical systems. Operational Framework Issue The Department does not have a signed copy of the operational framework. Observations – No signed copy of the operational framework The operational framework is a key component of the Department’s contract with the consultants. It describes the management practices, workflows and responsibilities used to fulfill the contract between the Department and the consultants. We requested a signed copy of the operational framework on February 21, 2008, but the Department was unable to produce one. As we were preparing our draft report in October 2008, the Department emailed a pdf file of a signed document to us. It appears from the email, that the Department forwarded us a document it had obtained from the consultants not from its own files. From this we concluded that the Department did not have its own signed copy of the Operational Framework. Since this is a key document in how the contract is managed, the Department should have had its own signed copy. We recommended The Department should ensure it has a signed copy of all contracts and supporting documents. Work of Audit Services Objective * To ensure the reports and recommendations of the Audit Services Unit (Audit Services) are adopted by the Department. * To ensure Audit Services focuses on areas of risk in the Department. Implementing Audit Services recommendations We reviewed two reports from Audit Services each of which had numerous findings and recommendations. The reports related to Long Term Care – In- Home Services and Standard Family Contribution. In our sample testing, we found a number of the same issues noted by Audit Services in its work. Audit Services had made recommendations to the Department for improvements in these areas. We believe, however, that the recommendations have not yet been implemented by the Department. The Department has an Audit and Evaluation Committee in place to deal with the recommendations of Audit Services. The responses from the Committee, as well as the implementation of the recommendations, are not always timely. Audit Services told us it has yet to receive a response from the Audit and Evaluation Committee on its 2006 and 2008 reports. We believe the work of Audit Services provides a valuable service to the Department and the recommendations of this section often provide significant cost saving opportunities to the Department. The timely implementation of Audit Services recommendations is beneficial to both the Department and the taxpayers of New Brunswick. The Department should ensure that a timely process is in place to implement the recommendations of Audit Services. Audit Services area of work We were pleased to see that Audit Services had focused some of its resources in the area of long-term care. Given the significance of its findings and the significance of the dollars involved, we believe Audit Services should focus some of its resources on NBFamilies each year. We agree with Audit Services assessment that the area of long-term care has numerous risks to the Department. We believe these risks could be reduced by establishing a regular audit of this area by the Audit Services unit. We recommended We recommended the Department implement the recommendations of Audit Services on a more timely basis. We recommended Audit Services incorporate in its annual audit plan one or more of the departmental programs processed by the NBFamilies system. Part C Results from Sample Testing Overview Introduction This part discusses our findings from our sample testing. Contents This part contains the following topics: Topic See Page Financial Assessments Not Completed 163 Contracts Not Signed 165 Supporting Documentation Does Not Backup Payment 166 Services Vary By Region 168 Service Requisitions Not Completed Appropriately 169 Payment for Clients Whose Level of Care is Unknown 171 Inspection of Long-term Care Facilities 172 Client Reviews Not Completed 174 Financial Assessments Not Completed Objective To ensure financial assessments are completed regularly and to ensure the amount of client contribution is correct. Sample information We selected a sample of 23 long-term care clients and 1 alternative family living arrangements client from the NBFamilies system. We tested to ensure that an up-to-date financial assessment was on file for each client and that the amount of client contribution was correct. Observations – Financial assessments not completed on a timely basis Of the 24 clients we tested, three clients did not have financial assessments completed on time as required by departmental policy. Two of these clients’ financial assessments were completed in 2005. Departmental policy requires staff to perform client financial assessments every two years unless the client is on social assistance. The financial assessments for these clients should have been performed in 2007. One of the clients is deceased as of June 2007. The third client’s financial assessment was completed in 2004. From our review of documentation, we believe the financial reassessment was done in 2006 based on 2004 data. Observations – Client contribution We also found nine clients where the client contribution amount was incorrect. As mentioned above, client reassessments are only performed every two years. The NBFamilies system automatically adjusts the client contribution for clients who receive the maximum amount of Old Age Security (OAS) or Guaranteed Income Supplement (GIS). However, for those clients who receive less than the maximum OAS and GIS and who receive CPP, the NBFamilies system does not adjust the client contribution. Therefore, the amount of the client contribution for these clients is incorrect when the rates change each quarter. This observation was noted by the Department’s Audit Services unit in its “Standard Family Contribution” report. Audit Services recommended that client income amounts be adjusted every time there is an increase in OAS/GIS and CPP. Audit Services also gave an example of a region that had a process for updating OAS/GIS and CPP income when an increase occurs and it recommended that the region be considered a model for all regions. Continued on next page Financial Assessments Not Completed, Continued Observations – Client contribution - continued Audit Services also recommended that financial reassessments be done based on a risk model. For example: (1) clients who receive OAS/GIS only, no reassessment would be required; (2) clients who have indexed pensions, reassessments be done yearly; and (3) clients who have non indexed pensions and investment income in addition to federal pensions be reassessed every 2 years. We thought both of these recommendations had merit and would help the Department in ensuring all clients are treated consistently and fairly. We did wonder, however, on the risk model for financial reassessments, if the Department should consider some minimal random checking of clients who receive OAS/GIS only. Observations – Filing of financial assessments In some cases financial assessments were difficult to locate. These assessments appeared to be stored in a separate area from the other client information. The Department was not surprised with this finding. We were told that with the implementation of “Front-end Integration” the person who performs the financial assessment could be different from the person performing the long-term care assessment and the information could be stored in separate locations. We believe information on a client should be stored in one place, either in an electronic and/or a manual file. We recommended We recommended the Department complete financial reassessments on time as required by policy. We recommended the Department obtain updated client financial information when completing financial reassessments. We recommended the Department review the recommendations of Audit Services and develop a process where client contributions can be updated when client income changes. We recommended the Department store client information in one location that is easily accessible to departmental staff. Contracts Not Signed Issue During our testing, we found two cases where payments were made to service providers who did not have a signed contract with the Department. First case Discussion with the region indicated that the service provider would not sign an agreement with the Department in 2008, but the Department continued to use the service. However, the hourly rate paid to the service provider was the standard rate paid by the Department to other similar service providers. Second case In the second case, the Department informed us that this particular service provider is used by all regions and typically each region has a contract with this service provider. This year the Department considered having one contract for all regions; however, no decision was made to do this. Consequently, no contract was in place for the period April 1, 2007– March 31, 2008. Our concerns We believe payments should not be made without signed contracts. We were told that for the second case various hourly rates were paid to this service provider depending on the negotiation by the social workers. We believe, had a signed contract existed for this service provider, consistent rates would have been used by the social workers. We recommended We recommended the Department only make payments to service providers who have a signed contract with the Department. Supporting Documentation Does Not Backup Payment Objective To ensure payments to clients and service providers are supported by invoices and supporting documentation. Background In NBFamilies, payments are made to service providers in two manners – electronic invoicing and traditional paper invoice submission. With electronic invoicing the following process is followed: * Service providers electronically submit invoice lines to the Department. * The NBFamilies system matches the electronic invoice information with a service requisition previously entered by a social worker. * If all criteria match, a payment is made to the service provider. * The service provider is required to maintain manual supporting documentation and if asked, provide copies to the Department. Supporting documentation did not agree with payment As part of our sampling, we asked service providers to provide us with supporting document for their electronic invoices. In our work, we found one case where the supporting documentation submitted by the service provider did not agree with the electronic invoice line that it had submitted. The supporting documentation totaled 199 hours but the service provider had submitted (and been paid for) an electronic invoice of 215 hours. The invoice was for the period of May 2007 for $2,808.06. The amount of the overpayment was $210.08. The Department informed us that it was going to set up an overpayment to recover the money. We are not aware if the Department is planning any additional action to prevent and/or detect future over billings by this service provider. No supporting documentation (subsequently provided December 12) One service provider was unable to supply the Department with any supporting documentation for an electronic invoice it submitted for the period of May 2007. The amount of the payment was $2,038.03. The Accounting Services section informed us that it was going to recover this money by withholding future payments to the service provider. Continued on next page Supporting Documentation Does Not Backup Payment, Continued Our concerns We are concerned that the Department has made payments that cannot be supported by backup documentation from the supplier. We would like the Department to devise a strategy to deal with service providers whose backup documentation does not support its electronic invoices. In the absence of other action, simply recovering overpayments does little to deter the service provider from submitting an incorrect invoice in the future. Ideas to prevent over billing could include selecting the service provider for extensive future audit or prohibiting the service provider from using electronic invoicing for future payments. We recommended We recommended the Department develop procedures to address cases where service providers’ supporting documentation does not agree with their electronic invoices. The procedures should have strategies to deter service providers from submitting invoices that are not properly supported by backup. Services Vary by Region Issue The types of services offered to clients can vary by region depending on budget restraints. Details In our sample we examined a file for a client who was receiving a particular service. The social worker we spoke with regarding this client believed that the Province had changed its policy and it no longer provided this type of service to clients thus the client was not eligible to receive the service. From our discussions with staff at head office, we learned that this particular service had not been disallowed by the Province and that regions can decide on the types of services they will provide to its clients. Depending on its budget, a region could disallow a service that is provided by other regions in the Province. We are concerned that the types of services offered to clients is a regional rather than a Provincial decision. We believe that this will result in inequities in the services provided to clients depending on where they live in the Province. Head office staff indicated that they were working on a “Menu of Services” project which will help to standardize the services provided to clients across the Province. We recommended We recommended the Department review its policy of allowing regions to decide on the types of services offered to clients. Allowing regions this level of decision making promotes inequity of services depending on where clients live in the Province. Service Requisitions Not Completed Appropriately Issue Social workers are not always completing service requisitions appropriately. Observation – Rate type of “none” In our sample, we noticed one payment where the social worker did not enter a rate type in a service requisition for a client. A service requisition is the way a social worker sets the limits for the types of services the Department will provide to a client. In this case, the service requisition indicated that the client was eligible to receive 1 unit of service with a rate type of “none” resulting in a total monthly cost of $2,040. Typically, the unit of service is entered with a rate type of hours or days, etc. (For example, 25 hours @ $8.00 per hour.) From our discussions with the social worker, she believes that the client was eligible to receive 255 hours @ $8.00 per hour for a total cost of $2,040. The client invoiced the Department for 201 hours @ $7.00 per hour AND 21 days @ $30.00 per day for night care. Total of the invoice and the payment was $2,037. Completing service requisitions with no rate type, allows the clients to determine the amount and type of services they will receive. The client in our sample paid the service provider an hourly wage less than the approved hourly departmental rate. The departmental rate requires homemakers to be paid a rate between minimum wage and $8.00. Minimum wage at the time of this payment was $7.25 per hour. This service provider was paid $7.00 per hour. Also, the client received a service that we were told was disallowed by the region. Continued on next page Service Requisitions Not Completed Appropriately, Continued Discussion with department From our discussions with staff at head office, we were told that they strongly discourage the use of 1 unit with rate type of “none”. However, it is not possible to have system edits to restrict its use because in some circumstances this requirement is needed. We believe reports could be generated to identify service requisitions using the 1 unit rate type of “none’. A review of these requisitions would identify social workers who are completing service requisitions in this fashion, as well as, identify clients who could be receiving services different from those approved by the Department. We noted that Audit Services also found this in its work on Long-Term Care In-Home Services review in 2006 and made a recommendation in this area. We recommended We recommended the Department investigate service requisitions entered without a rate type to ensure the clients are receiving the approved level of service. We recommended the Department instruct social workers who use this system function that, except in certain circumstances, it is not an acceptable way to enter in a client’s level of service. Payments for Clients Whose Level of Care is Unknown Issue One payment related to a client who has a very complex case. The level of care required for this client was unknown. The social worker indicated on the service requisition that the service provider would contact the Department each month and inform them of the amount of payment required for the month. The payment amounts for three months were in excess of $25,000 for each month. When we obtained a copy of the invoice, we found the Department was charged 1,400 hours of care for this client at an hourly rate of $15.48. This payment was made by electronic invoicing. The Department did not review the invoice prior to payment. The social worker entered the cost into NBFamilies and the system paid the invoice. During our audit, we contacted the supervisor who agreed that the hours seemed high for one month of care. After receiving a copy of the invoice from us, the supervisor investigated the payment and determined that the level of care was reasonable given the client circumstances. Discussion with department Head Office informed us that this is not a normal case. This client is a special case and needs constant care. Finding someone to care for this client was a challenge for the Department and the payment method was secondary to finding proper care for the client. Even though we understand the concerns of the Department, we believe that someone within the Department should be accountable for the payment. In cases such as this, the Department should request the service provider to submit manual invoices that can be reviewed and approved by appropriate levels of management within the Department. Head Office asked the client’s program delivery officer to review the case. We were informed that the NBFamilies requisition will be updated to more clearly reflect the level of care required and manual invoices will be requested from the service provider and reviewed for reasonableness prior to payment. We recommended We recommended that when the Department is unable to determine the level of care needed for a client, the service provider should submit manual invoices to the Department that can be reviewed for reasonableness and approved by the various levels in the Department. Inspection of Long-term Care Facilities Objective To ensure Adult Residential Facilities (ARF) are properly inspected as required by departmental policy. Background As part of our long-term care testing, we reviewed ARF inspection documents for clients in our sample to ensure that the facilities were licensed and eligible to receive payments from the NBFamilies system. We audited ARF inspections in 2005 and made a number of recommendations relating to the inspection process. We followed up on these recommendations last year in our annual Report. See our follow up chapter for more information on the status of these recommendations. One of our 2005 recommendations was for the Department to develop a standardized inspection form to be used by all regions. The Department implemented this recommendation in January 2008. We reviewed the Department’s use of the new form to ensure all regions were using the new form and that they were completing the form properly. Testing details We tested 18 ARF inspection files – 11 facilities were inspected before January 2008 using the old inspection form and 7 facilities were inspected after January 2008 when the new form was implemented. Findings – new form not used when required In the Moncton region, five facilities were inspected after January 2008. In all five cases, the new inspection form was not used when required. The Department indicated that this was an oversight and the region is now using the new form. Findings – inspection not completed properly A new facility was inspected on October 31, 2007. The old inspection form was used but it was not completed properly. Two pages of the old inspection form were not completed as required. The Department did not comment on this finding. Continued on next page Inspection of Long-term Care Facilities, Continued Findings – five inspections in one day Inspection documents indicate four inspections were done on the same day, at different addresses, by the same inspector. Given the process required to complete an inspection, properly completing four inspections on the same day would be very unlikely. Also, licensing documentation for a fifth facility was dated on the same day, but no inspection documents were present. The Department explained that these facilities were small with few beds. A central office manages the administration of the facilities. The inspector reviews the central files and then visits each facility. Findings - temporary certificate not issued Inspection documents indicate that the facility was to receive a temporary license for three months. The facility received a permanent license for one year instead. The inspection documents indicated that the employee files at the home were incomplete and a permanent license should not have been issued. The Department did not comment on this finding. We recommended We recommended the Department ensure all regions are using the new form and the new form is completed properly. We recommended the Department only issue a permanent license to an ARF if the facility meets all the standards required in the inspection documents. Client Reviews Not Completed Issue Clients are not being visited on a regular basis. Observations – Client not visited regularly We reviewed payments from the NBFamilies system for 24 clients and make the following observations: * One client’s last assessment was in 1996. From our review of the client’s file on NBFamilies, there was no evidence in NBFamilies that the client had been visited since NBFamilies was implemented. The electronic case plan was dated in 1986. There was no evidence that the plan was revised or updated since that time. * One client’s assessment and case plan was completed in 2000. This client’s financial assessment was completed in 2004. This provides evidence that the client was contacted since the time of his last assessment. From our review of the electronic file, there was no evidence that the client had been visited or a reassessed since 2004. * One client had not been visited since 2004. The current social worker reviewed the information recorded in NBFamilies and saw no evidence that the client had been visited since the initial assessment in 2004. This client had numerous problems with her file that have been discussed through out this report. Department policy At the time of our audit, the Department did not have a policy on how often client reviews and/or reassessments should be completed. However, the practice followed by Long Term Care staff was that reassessments would be undertaken when the needs of the client significantly changed, when the client was going from in-home services to a residential placement or when the client him/herself was requesting a reassessment. In July 2008, the Department developed guidelines suggesting that an annual case review be conducted for Long Term Care clients at home or in an Adult Residential Facility. To conduct the annual review, the social worker personally contacts the client and asks a number of questions relating to the client’s condition, the adequacy of the services, the client’s satisfaction and the client’s financial situation. Depending on the client’s response a reassessment may be performed or the service plan may need to be reviewed. Continued on next page Client Reviews Not Completed, Continued Our concerns We are concerned that some clients have been receiving services for many years without having regular contact with the Department. We hope the new guidelines will eliminate this problem. We question whether guidelines with suggestions are strong enough to ensure client reviews are performed. Should this be a policy of the Department with a requirement that reviews be completed? We expect the client reviews to be documented in NBFamilies to show evidence that the client review was completed. Conducting the annual client review will help ensure that the social worker does not automatically click the NBFamilies review task. We will monitor this in future audits to ensure social workers document the annual client review. We recommended We recommended the Department conduct client reviews on a regular basis. The client reviews should be documented in the NBFamilies system as evidence that the reviews were completed. We recommended the Department review the guidelines with suggestions to determine if these suggestions are strong enough to ensure social workers have contact with clients on a regular basis. Part D Accounting Issues Overview Introduction This section discusses accounting issues we noted in our testing of NBFamilies Contents This part contains the following topics: Topic See Page Accrual of Unbilled Services 177 Accrual of Unbilled Services Issue The Department does not accrue for services received by clients by year end but that have not been billed by service providers. Process The following process is used to request services for long-term care clients. * Social worker determines the level of care required and completes a service requisition. * Service provider provides care to clients. * Client or service provider submits invoice for payment. Cutoff issue At year end, many clients have received services but the service providers have not billed the Department for payment. Since these services were provided to clients in the period April 1 to March 31, they should be recorded as expenses in the fiscal year the service was provided. The Department does analyze payments made to service providers from April 1st to the year end cutoff date (April 25 in 2008) and the Department accrues the services that relate to the prior year. However, the Department does not estimate and accrue the amount of services that have been provided but not billed by service providers. We analyzed NBFamilies data to estimate the amount of unbilled services for the 2008 fiscal year. Based on information obtained from the Department, we estimate these services are in the range of $5 million. Even though this amount is significant, some of it could be offset by the amount of unbilled services that relate to the 2007 year but were paid in the 2008 year. The netting of these two amounts could significantly reduce the amount of the cut off error. However, changes in billing rates and the speed of suppliers submitting their invoices would cause fluctuations in the accrual amounts between the two years. Discussion with Department We discussed this issue with the Accounting Services section in the Department. The manager of this section was going to perform his own analysis of the payments for a fiscal year to produce his own estimate of the amount of unbilled services. The manager was hesitant to accrue expenditures based on estimates for budgetary reasons, however, he indicated he was willing to track expenditures after the cutoff period and accrue actual payments. Continued on next page Accrual of Unbilled Services, Continued We recommended We recommended the Department estimate and accrue the cost of services that have been received by clients at year end but have not been billed by service providers. Section B Department of Social Development NBCase Expenditures Overview Introduction The purpose of this section is to discuss our findings and to make recommendations from our audit of NBCase expenditures. In this section This section contains the following topics: Topic Page Access Controls - Terminating Users’ Access 180 Audit and Evaluation Committee 182 Improvements in the Follow up of Regional Investigators’ Work 184 Improvements in the Timely Completion of Case Reviews 186 Access Controls – Terminating Users’ Access Issue During our testing of the NBCase system, we found that 27 NBCase user accounts had not been terminated after 90 days of inactivity. Terminating user accounts in a timely manner reduces the risk of unauthorized access to information stored in the system. Results of Testing The table below shows the length of account inactivity and the number of users who had not accessed the system in that amount of time. Length of Account Inactivity Number of Users 1 - 3 years 8 7 – 11 months 10 3 - 6 months 9 Discussion with Department When we discussed this issue with the Department, we were given the following explanations as to why user accounts were not terminated. Number of Users Explanation from Department 14 RUSAs did not send a termination request when required. 7 Supervisors did not send a termination request for users who were on sick leave and maternity leave. 6 Supervisors did not send a termination request for users who no longer needed access to the NBCase system. From our discussions, we learned that the majority of the Regional User Support Analysts (RUSA) (who are responsible for terminating system access) are relatively new and may not be aware of the requirements and the process for terminating system access. Provincial Standards In March 2003, the government released “Password Standard for User Accounts”. These standards require user accounts to be terminated if they have been inactive for 90 days. We recommended We recommended the Department terminate inactive user accounts after 90 days of inactivity. Continued on next page Access Controls – Terminating Users’ Access, Continued We recommended (continued) We recommended the Department train Regional User Support Analysts and supervisors so that they are aware of the process for terminating system access. Departmental Response The department agrees with the recommendation to disable NB Case user accounts after 90 days. As of November 2007, NB Case access is tied to Active Directory. With Active Directory, sign in accounts are disabled after 30 days of inactivity; so technically, an NB Case user cannot access NB Case after the Active Directory account has been disabled. Reports are produced monthly and forwarded to supervisors to terminate system access when necessary. This process should address this concern. Audit and Evaluation Committee Introduction The Department of Social Development has an Audit and Evaluation Committee in place whose purpose is to act as a decision making body and to provide leadership and support to the Audit Services unit and the Planning, Research and Evaluation Branch. Audit Committee – Promotes Strong Control The Department’s Audit and Evaluation Committee has several key roles and responsibilities outlined in its terms of reference. These include : * identifying areas of concern of a financial and program nature that could benefit from an independent review/evaluation, * focusing attention on the audit functions such as the effectiveness of management systems, compliance, coordination of internal audits, accountability, and follow-up of both internal and external audit recommendations, * reviewing and approving the annual Internal Audit Work Plan and the Evaluation Work Plan, and * accepting or rejecting the Audit or Evaluation recommendations based on the findings in the audit or evaluation report. In addition, the terms of reference state the Chairman (Deputy Minister) will be responsible to follow-up with lead Assistant Deputy Ministers as to the development and status of the implementation plans. The Terms of Reference also requires the Committee to meet every two months for a maximum of two hours. This type of senior management support of internal audit’s work cannot help but strengthen the overall control environment in the Department. We are pleased to see a committee such as this in place in a government department. Frequency of Meetings During the past two years, we noticed that the Audit and Evaluation Committee is not meeting bi-monthly as required by its Terms of Reference. The Committee only met two times in each of the last two years. Since we believe the existence of this Committee is beneficial to the Department, we recommended the Department continue in its support of the Committee and its ongoing efforts. In order to support a strong control environment, however, we believe the Committee should meet regularly as outlined in the Terms of Reference. Regular meetings will provide the direction needed to allow the Committee to be effective in carrying out its responsibilities. Continued on next page Audit and Evaluation Committee, Continued We recommended We recommended the Audit and Evaluation Committee should meet regularly as required by its terms of reference. Departmental Response Thank you for the acknowledgement of the benefits of the Audit and Evaluation Committee however the bimonthly frequency of this Committee has proven to be unrealistic for senior management availability. The Terms of Reference have been revised so that meetings will be held four times per year. Improvements in the Follow up of Regional Investigators’ Work Introduction The purpose of this section is to update our findings on the work of the Regional Investigators’ Supervisors. In the past two years, we reported problems with the follow up performed by the Regional Investigators’ Supervisors. We noted that the Supervisors were only following up on the Regional Investigators’ work in 47% of the cases. Definition Regional Investigators: investigate social assistance clients based on tips that the Department receives from various sources. Regional Investigators’ Supervisors: follow-up on the recommendations of the Regional Investigators after six weeks to ensure the Case Managers have made the recommended changes. Audit Findings Again this year, we reviewed the work of the Regional Investigators’ Supervisors. We found for the period April 2007 – October 2007, Regional Investigators recommended changes to social assistance payments in 33% of their investigations. Regional Investigator Supervisors followed up on 69% of these changes. This is a significant improvement over the 47% figure for the last two years. We make the following comments from our review. * We are pleased to see that the Regional Investigators’ Supervisors have improved in completing their follow up tasks. * We would like to see the follow-up percentage increase even more in the future. * We note that the smaller regions have a lower follow-up percentage than the larger regions. * We learned from the Department that it periodically monitors the work of the investigators but it was uncertain why two regions had low follow-up percentages. Continued on next page Improvements in the Follow up of Regional Investigators’ Work, Continued Observations From our work, we believe the Department should continue to educate the Regional Investigators’ Supervisors so that they are aware of the importance of following up on the work of the Regional Investigators. The Department should continue to monitor the Regional Investigators’ Supervisors and determine why some of the supervisors are not completing their follow up. Implications As we mentioned last year, a client could continue to receive ineligible payments if the Regional Investigator’s Supervisor does not verify that the Case Manager has reviewed and implemented (when justified) the recommendations of the Regional Investigators. Conclusion The Department has made progress in implementing our recommendations of the past two years. We believe the Regional Investigators’ work is a key control and that timely implementation of their recommendations could produce cost savings for the Department. We will continue to monitor the work of the Regional Investigators and their supervisors in future audits. Departmental Comments Thank you for the acknowledgement of our significant progress in this area from 47% follow up in the past two years to 69% in the current year. We will continue to monitor this work especially in the regions with the lower percentages to achieve increased percentages that could produce cost savings for the Department. In addition, there are a number of changes proposed for the March 2009 release of NB Case that will address some of the data issues and will no doubt result in improvements. Improvements in the Timely Completion of Case Reviews Introduction The purpose of this section is to update our findings relating to the completion of case reviews. As in the past, we divided this section into two parts – case reviews for alert clients and case reviews for non-alert clients. Definitions Case review: A case review is a process where a departmental employee visits a social assistance client to verify client information and determine that the client is still eligible to receive social assistance benefits. Case reviews are often performed during the summer months by summer students. Alert client: Clients are given an “alert” status if they are considered to be violent. Case Managers are required to perform case reviews for these clients. Continued on next page Improvements in the Timely Completion of Case Reviews, Continued Alert Clients We previously reported that case reviews are not always completed on time for alert clients. The following table shows the progress that the Department has made in completing case reviews for alert clients over the past few years. Alert Clients Year Case Review Required 2007 Overdue Reviews 2006 Overdue Reviews 2005 Overdue Reviews 2000 0 0 3 2001 0 0 2 2002 0 5 18 2003 0 2 11 2004 0 5 14 2005 1 4 12 2006 0 25 NA 2007 14 NA NA Total Overdue 15 41 60 Total Number of Alert Clients 589 586 563 Percentage of Alert Clients with Overdue Case Reviews 2.5% 7.0% 10.7% From the table, we can see that except for one client, the Department has completed the older case reviews for the alert clients. The percentage of alert clients with overdue case reviews has dropped to 2.5% from 10.7% in 2005. Continued on next page Improvements in the Timely Completion of Case Reviews, Continued Non-alert Clients We previously reported that case reviews are not always completed on time for non- alert clients. We also noted a number of case reviews were more than one year old. The following table shows the number of overdue case reviews for the past few years, as well as the percentage of overdue reviews. Non-Alert Clients Year Case Review Required 2007 Overdue Reviews 2006 Overdue Reviews 2005 Overdue Reviews 2000 0 0 0 2001 0 0 1 2002 0 1 3 2003 0 0 19 2004 0 13 101 2005 2 84 584 2006 7 644 NA 2007 443 NA NA Total Overdue 452 742 708 Total Number of Non-Alert Clients 22,997 23,638 24,527 Percentage of Non-Alert Clients with Overdue Case Reviews 2.0% 3.1% 2.9% From the table, we see that the Department has improved significantly over the past few years in completing the case reviews. Of the outstanding case reviews for the current year, we note that 382 are due after August 2007. Since the majority of case reviews are completed in the summer by summer students, these cases will not be completed until the following summer. This is an inherent limitation in having summer students perform case reviews. Conclusion The Department has made progress in implementing the recommendations that we made in the past two years. As we believe case reviews to be a key control area, we will continue to monitor case reviews in the future to ensure they are completed on time. Continued on next page Improvements in the Timely Completion of Case Reviews, Continued Departmental Comments Thank you for the acknowledgement of our progress in the area of timely case reviews. The report is divided in to two parts: A) Alert clients We have made a concentrated effort in this area and the outcome is the lowest percentage in [overdue] case reviews for alert clients for years. As noted, the percentage has dropped from 10.7% (60 clients) in 2005 to 2.5% (15 clients from a total of 589) in the reporting period. We will continue to communicate the importance of completing these reviews in accordance with our policies; however we must be cognizant of safety and security of staff for those few alert clients who could pose a challenge. B) Non alert clients We have also made improvement in the area of non alert clients. Staff has completed 98% of case reviews this past year as compared to 97% from the previous audit. Matters Arising from our Information System Audits Chapter 8 Chapter 8 Matters Arising from our Information System Audits 180 Report of the Auditor General - 2008 181 Report of the Auditor General - 2008