The fundamental way of banking undergoes a rapid transformation in the world of today in response to the forces of competition, productivity and efficiency of operations, reduced operating margins, better asset/liability management, risk management and Anytime and Anywhere banking. The technology has changed the face of banking by increasing high level of competition in the market together with increased awareness and quality consciousness among the customers by increasing the transparency in service. It is due to rapid development in technology that there is a sea change in both retail and investment banking. During 1980s the Rangarajan Committee report gave the first blue print for computerization of banks in the country. The Banks have been equipped with variety of platforms related to hardware, operating system, software and application software has implemented with different platforms. It is essential to have interface with different systems and other bank network environment. The cloud computing in banking sector aims to provide better and more innovative services to customers. The advancement of service oriented architecture (SOA) has brought all the banks to come closer to establish and run a virtual business with online services. Cloud computing implements SOA in which IT resources are offered as a service that is more affordable, flexible and attractive to banking sector. This article discusses the benefits and challenges of cloud technology in banking sector and conceptual architecture for a bank operation environment.