Problem Statement: I/O performance ....database queries,

                         performance ...web hits ...web-server-responses ...back-end storage,

                         
                         peformance optimization by introducing one layer where performance can be maximized or
                         optimized by having a appropriate [dynamic] storage provisioning(storage virtualization) 
                         ...viz. no of I/O hits, storage space requirement(lun size or filesystem size)
                    
                        further as published in the document projects.pdf ( 175), 182), 184),  187),253), 254)!!, 426),591),798),
                        778),797), 847), 878)
                        and else where by this individual.


                      !!  one of the reason's why RAID and storagemanagement is better-off automated is , in a one off a kind of
                        assignment i took-up in the recent past ....a fairly accomplished sysadmin with 15 - 20 yrs of experience
                        on unix systems ....when monitoring and managing lun's ...is counting up 'iostat' stat's on virtual lun's
                        carved out of a RAID ...storage-array!! ...precision ...ability to armour ... a sysadmin with kind of ability
                        fine tune ....performance!! engineering!! ....given that storage forms the foundation based on which the
                        performance of the most application's host or making use of storage-space from such raid solutions.  !!,
                        while a iostat can still reveal the fact that RAID solution or virtual lun is at best skewed.


                        virtualize RAID, volume-manager, virtual file-systems, serveral other use-case modelled functionalities at lun level 
                        ....in high-end storage box's or array's / SAN environments ...this is different or extends beyond what was or is 
                        the functionally feasible on storage array's like  sun a1000 , a5000 and also effectively utilize the processing
                        power.

                        Automation inside the storage-box-array/storage behind SAN-fabrics by encapsulating innovative set of layer's of software
                        viz. RAID/volume-management/filesystem-management/use-case-modeled...software stack...compression/backup...etc

Opportunity a:)   Refer to the document opportunity_innovative_web_token_session_mgmt.txt,
                                                    opportunity_webapplication_performance_tuning.txt,
                                                    opportunitydbenhancementsperftuning.txt,
                                                    

                   By understanding the computing virtual architectures(2tier or 3tier ...differentation in various
                   layers or categories in software stack viz. application-processing ...databse-processing),

                   load balancing of traffic can be done at  applications-layer ( both applications layer ...websevers,
                   email, group-ware ,file, database servers) 

                   ...a hidden layer can be introduced ...which at storage layer...where I/O through puts can be optimized.
                  by having the storage for webservers, database servers ...with RAID ...and volume-manager for
                  storage  managing storage in  high-end storage-arrays or  storage-area-networks 
                  ...with the servers provisioned with sufficient bus-width/bus-speeds or
                  bandwidth ...to these storage.

.
Opportunity b:)  Performance engineering of IO ...metric's ...efficiency based the usage of the lun viz. optimized for writes / reads

                   deriving from RAID ...mirroring /concatenation ...a concepts used in volume-manager
                      embedding or bundling the concepts of RAID with highend-storage-arrays or SAN...,
                  
                     ...luns allocated to a host can be managed virtually ...based on the I/O hits , data-usage.

                     where in the embedded software layers in storage-arrays or SAN determine the
                     layout as also architecture of the VIRTUAL lun based on the RAID principles, [filesystem management]

                    Eg: based on the I/O hits ...additional mirrors can be provisioned in SAN/Storage-array,
                    
                         based on the present  space(empty...expansion principle) requirement ...can be provisioned 
                         dynamically  by (...allocating a larger virtual lun) or ( by changing the layout of the lun using RAID) .


               while raid 1+0 with number of mirrors increased proportional to i/o hits or traffic offering best layout,

               simulate raid 5 with mirror's for all disk participating in the raid 5 , 1+0 for the parity disk in raid 4,
               choosing the best layout...also offering redundancy based on i/o and redundancy requirements
               -----------------------------------------------------------------------------------------------------------------

               Ability to implement RAID on a bunch's of mirrored disks (vi. each disk participating being virtual disk) ...interpret

               [ analyze ...vxiod daemon in the vertias suite ...buffer the io to multiple disks simultaneously...multiplexing]
               -------------------------------------------------------------------------------------------------------
               =======================================================================================================
 
               thus the scope of implementing raid 5 or raid 4 on a mirror ...given the scope or context that io can be
               buffered to multiple disk's at the disk driver level.

               a disk being the basic unit or building block in standard raid definition, the concept of building block for
               raid i.e. disk can be redefined ...thus a virtual disk (1+0 or 'n' number of disks as required for redundancy
               and performance) can be virtuall defined as single disk a bulding block and used in raid layout's.

               a different kind of definition for ...what has traditionally been defined for plex or out look implementing
               the raid and plex's ...as can be defined in raid concepts ...and taking veritas raid implementation or definitions
               as a anology ...refer to how ods in older versions of solaris is implemented ...the same envisaged at the level
               of a disk as the basic minimum building block instead or inlieu of a disk partition....to leverage full potential
               of the RAID ....viz. raid 0 being to do with disks ...ability to bunch disks ...buffer the io on a virtual disk,
               to leverage the capabilities of the RAID  and its various kinds of implementations.

               example scenario being potential ability to implement a radi 5 on 3 (three) a1000 or a5000 arranged or connected
               to a OS host in series.
               ------------------------------------------------------------------------------------------------------------------



Opportunity c:)   further to opportunity b) ...a file-system could also be virtualized on the storage allocated to hosts ,
                       by shielding the OS layer ...viz. data-management of all applications ,...say based on
                       attributes settable by OS ...RAW or filesystem  ....viz. by bundling or embedding the filesystem
                       management principles or logic with SAN/Storage-array management logic boards or embedded OS,
                       with virtual luns marked for filesystem management.


                    a notable managable-attribute that can be set ...viz. when using a filesystem-lun ...space to managed (defragment
                    or re-organization of contigious-space allocation) by the host to which lun(say kind of database) is allocated
                    or array managed...a key factor for database performance tunining

                    [ ....by utilizing concepts of mirroring ...by potentially having a attribute 'quiecesing' interval/time-at-which
                          configurable through a management interface on the host to which the lun was allocated

                      deriving from how a bit based archiver or differential (dirty-page logging) ...efficiency in backups, 
                      how snapshot's work ....a virtually defragment lun introduced into a mirror ...which can be brough-online
                      and other copies in the mirror synchronized or where no provision for mirroring ...the non-defragmented
                      lun can be taken offline.
                    ]

                    ...given the scope of the virtual
                       space in virtual file systems (optionally veritas) ...how virtual contigous address space is generated ...viz.
                       i/o optimization ....virtual address generation of the data or rows in the filesystem for optimal tablescan 
                       i.e. ability to use concepts used b-tree or index or data hashing alogorithms ...as part of the virtual address
                       space generation in filesystem ...so as to max or leverage the advantages of b-tree loading of the node or data
                       retrival....refer to 'opportunity_innovativefoundation_4_ERP_MIS_accounting_applications.txt' and
                       'opportunity_dynamic_virtualization_storage_blueprint.txt'


                    eg:refer to the context ... 'opportunity_database_optimization_tuning_tip.txt' 

Opportunity d:) ability to increase the processing power of  SAN/Storage-arrays ...given that these are appliances ,
                      caterin to specific needs ...thus the ability to add additional processing power by hot swappable
                      or pluggable processors ...much like in high-end computing arcitectures ...given the context of
                      
                     1) Manage more no of  virtual-luns , hard-luns ...additional storage provisioned into a SAN/storage-array.
                     2) refer as specified in Opportunity b) & opportunity c).

opportunity e:) with reference to opportunity b) and opportunity c) as mentioned above ...a protocol or standard
                     ...viz. based on xml-rpc or com/dcom or corba ...which gives the application software ...viz. mail-servers/
                     database-servers/group-ware/ ...a interface to interact with embedded-software-layer containing or
                     maintaining RAID and filesystem-management on the luns being used by their specific host for exchange
                     data specific to metrics and storage....viz. requirements like provisioning of a lun , rawspace or filesystem-space,
                     preset no of  mirrors for I/O balancing or a host communicated dynamic number of mirrors for a lun allocated to
                     a host ...over-ridable or not over-ridable, [ backup-frequency of the lun, snapshot
                     ...frequency of the lun ...refer to projects.pdf & else-where as envisaged by this individual , remote replication of
                     lun ...say compatible appliance installed with sufficient bandwidth ], ...use-case modelled list of interfaces, 
                     and functionalities to be implementable.

                    while it may be remembered that each virtual-lun mapping to at least one physical-lun i.e. hard-disk gives
                    ideal or optimal results ...as sharing a physical-lun by one or more virtual luns can potentially result in I/O
                    hindrance.

 
                   
Opportunity f:) with management interface ....ability to enable/disable by admistrator/interface that lets these attributes to be 
                managed or set at the OS or application level to which a lun is allocated from the storage box ... various attributes
                applicable or configurable at lun level.

                apart from below ...this include's ...instead of a storage administrator carving out a lun and allocation to host,
                the management interface on the host ability to requisition or provision for a lun of specific size to the storage-array/SAN,
                which in effect also keep's or take's into account as discussed in b) above.
                

                eg:

                content versioning ...could potential be made part of such storage-box or array with addon logic-card or customizable
                feature(optionally loadable kernel module) on such a storage box or container.

                specific scope for feasibility being deriving from how compression works ...unix...diff ...utilities ...efficeint
                storage and organization of content on such luns.

                also given the context of patented concepts like bit-based archiver / bit-based flash-backups(plex or volumes)
                as also data backup to tape mediums...superior backup solutions (may refer products and marketing literature for
                backup solutions from ibm,veritas,legato )  etc.

                 Deriving from how hardware-compression is handled in storage backup solutions,
                 similarly ...[ a generic-compression-card ]...can be part of the storage-box,
                 that can be utilized ...when a attribute on a lun is enabled for data compression.

 

                refer to the document 'opportunity_DWH_BI_product_orchrestrization.txt'


Opportunity g:) deriving from how grid computing works (eg: sun fire servers, e10k....ibm lpars...)

                     ....a innovative storage-box or container or array deriving from above scenarios or business opportunities

                     ....a dynamic algorithm for grid computing ...that allocates sufficient processing power and resources
                         to a specific task ....applying use-case modelling makes the envisioned sketch ...into a feasible solution.

                     
    
Eg : usage case scenario or potential scope of above envisaged solutions

       file-servers as apache+http_fileserver over volume-manager + [virtual file management solutions]
      highend storage in back ground using iscsi or gigabit-ethernet ,

      storing or recording GPS data-streams or GPS feeds, similarly recording stock market feeds


     also refer to  opportunity_innovative_web_token_session_mgmt.txt,
                    opportunity_integrated_EDI_fax_scanner_portable_computing.txt,
                    opportunity_datastorage_mediums_data_access_views.txt,
                    opportunity_innovative_directory_servers_views.txt.

Opportunity h:) deriving from opportunity c) and other context above

 ...given the scope of the virtual
                       space in virtual file systems (optionally veritas) ...how virtual contigous address space is generated ...viz.
                       i/o optimization ....virtual address generation of the data or rows in the filesystem for optimal tablescan 
                       i.e. ability to use concepts used b-tree or index or data hashing alogorithms ...as part of the virtual address
                       space generation in filesystem ...so as to max or leverage the advantages of b-tree loading of the node or data
                       retrival....refer to 'opportunity_innovativefoundation_4_ERP_MIS_accounting_applications.txt' and
                       'opportunity_dynamic_virtualization_storage_blueprint.txt'

Opportunity i:) deriving from all of the opportunities as interpretable or envisable from above

                    read through the document "opportunity_client_side_server_side_validation_storage_into_repository.txt"

                ability to provision or usage operating-system-kernel-module for processing of data, before storing into the
                storage.


Opportunity j:)  





                  Deriving from grid-computing other similar architecture's in vogue,  much like the ability to allocate more
                  processing power to process'es requiring more processing power ...say analogous to thread priority,

                  ability to allocate more disk-block's (mirror's ...at a microscopic level that a lun) to logical data-block's
                  (or portions of a file-system) impacted by high IO ...without having to re-organize underlying physical-disk-layout
                  for the logical data-blocks-or-structure to achieve efficiency or performance in IO.


                  A filesystem is nothing but hashing of data on a data with a user-friendly api ...access such data.

                  A good understanding of Raid 0 (stripe size), virtual filesystems (vxvm), lvm, ods, vxvm,

                  ability to re-architect how disk IO is done at disk-driver level , embedded-firmware of storage box's.

                  Read through opportunity_datastorage_mediums_data_access_views.txt,
                                                               

Problem statement :

                Deriving from various realtime use-case scenario's ...ability to arrive at fine-grained control over
                the number of mirror's ...splintering or bifurcating IO's .....contagious....virtual-address-space.

                case-study: studying the disk-io blockage or hindrance of a DB2 instance on raid over scsi-disks in a ds4300 array
                            a 2000 odd users accessing ...a 3 tier application ...with majority of hits effecting 3-4 database objects,

                            ability to interpret and control some or more of the following ...the usage or interpretation of efficiency
                            from database i.e. data-structures of organization of data
                            while a physical contagious disk space ...can give efficient data-reads ...when implementing the concept of
                            striping ...ideally translate into efficient IO read's ...simultanous reads from disparte disks ....ability
                            read more data than from a single disk ...ability to leverage caching or physical-memory(RAM) for efficiency.

                            a Qmail application catering to 1500+ users on a single scsi-25-GB disk with a 5mb storage-folder-each,
                            with majority or concentration of the users in a few-specific folder's ...thus ability to identity
                            cause for IO hits or blockage...ability keep the logical-structure of the data intact and force a physical
                            placement a few logical structure entities(containers or directories) across spindles or relocate, control
                            the physical data placement.

                            how symbolic-links work ...thus the ability to chain disk blocks spread across the disks, innovation in
                            linking chunks of disk blocks ...understanding unix process communication architecture ...pipes...descriptors,
                            ability mirror ...disk-blocks ...that can fit in the given scope of the context detail in this section.

 
		    Disk I/O is constraint ...not envisaged at most  application software level in the stack of software.
                    ability to gain fine grained control over placement of folder/containers/directories on virtual-filesystem.


              Irrespective of the capacity of disk , RPM rate's offered ....given that number of  thread's /sessions seeking IO
              on the disk ...i.e. higher-level ...number of concurrent user's who can access data, use application-software
              without  performance hindrance.


              given that above is the case-scenario most application-software in the stack above a disk IO driver, wether email-application,
              or database.

              While having or following the advocacy of having a RAID solution with more mirror's would solve the performance-hindrance to
              a extent.

     Load-test-scenario 1: for instance qmail servicing SMTP , data hosted on the server with a single disk ...scaling up of users

                        2: given that databases require contigious disk-space for efficienct IO, data oraganization ...number of
                           requests from end-users to a object in database or number concurrent read/write's with-out hindrance
                           to disk IO performance.


             addressing the scenarios in  1) , 2) ...by employing RAID solution judiciously.


             ability to generate virtual-lun /virtual-filesystem ...by having the ability to define the number of mirror's, as
             also having the data spread across virtual-address-space i.e. interleaved or spliced or striped , ...by utilizing or deriving
             from the lvm(for instance aix/hp,ods-solaris,vxvm) concepts ....ability to define more number of mirror's to data-subsets
             (eg: in the filesystem or a database ...viz. a container or folder or directory ....given that specific data-sets
             experience's more number of IO hits, instead of having a homogenous i.e. uniform ...mirroring-level across the lun.

             
            feasibility given the scope , being subject to disk-geometry ...io plexing ...various attributes of disk's, arrays,
            putting place appropriate disk i/o device-driver at host level or embedded firmware in array other similar storage-box's,
            matching the objectives.
            -----------------------------------------------------------------------------------------------------------------------------
            
            Given that most the RAID carving or derivations, virtual-file-system are virtual entities in the sense of logical interpretability
            or tangible nature of their sense.

            the below envisages essentially how data should reside ...and a logical view of such data ...as a fallout of potential 
            advantanges or objectives to achieved ...keeping the integrity of the logical view of the data, writes on disk translating
            into data consistency and integrity ....traversal of filesystem.

            rationalizable concepts handing few concepts, trade-offs, advantages (mirroring) at logical lun-level or chunks of 
            file-system impacted by heavy io ....ideally all objectives envisage above should be in a automated manner or a
            high-level-advanced user or administrator of such data-repositories.

            
        Eg: deriving from how meta-data is stored in directories , super-blocks ...re-architecting the structure of meta-data,
            as also understanding how file's, folder's are linked in a file-system ....how various file-systems in vogue on various
            operating-systems have crash-proof or recovery of data is achieved, how fsck works or is implemented, in most filesystems
            only superblocks are replicated or duplicated or multiple copies present on the file-system for redundancy ....
            
            A customized or custom developed filesystem where data-blocks are also duplicated or replicated or multiple copies
            stored on the filesystem ...such that as envisaged various objectives defined above are fulfilled for instance
            when traversing a file-system ...from one data-block to next-data-block ....the addressing schema may be such that
            when traversing from one datablock to next data-block ...the pointer from data-block may contain or traverse to any
            of the copies(mirror's) ...of the subsequent datablocks.


	   conceptualizable idea: Begin by breaking the physical-disks into addressable optimized splice-size or block-size for
           or analogical concept: a ideal IO blocking factor....when implementing a address schema for a disk-space ...standard
                                  ufs  contigious-disk-space(viz interpretations of a disk-io seek() call) being the only means.
            
                                   data-hashing ...using a filesystem ...implemented as a hierarcial-tree ...at each node 
                                   ...branching into multiple-child-nodes ...at any given node-level representing data or
                                   copies of mirror's ...ideally carving into or disk-space on disk's or platter's across spindles,
                                   i.e.at any given node-level all child-nodes at the level are mirror-copies or themselves located
                                   on disparate disk space.

                                   Making directories or container's as analogous equivalent of a volume ...place's for holding
                                   key meta-data...various controlable features.

                                   Most file-systems are hirerarcial in nature ...ability to encode or use a relative addressing-schema,
                                   much on the lines how hireracy of folders or directories or chained to form a virtual view of a file-system,
                                   combining the features of virtual-file-systems where-in a folder or directory or container can grow
                                   or add more-space based on requirement, grows dynamically ...instead of standard addressing schema as found
                                   in the ufs ... address space always remains or appears contigious ...as a file grows so does its address-space,
                                   visualizing how a file IO is done ...traversal from one data block to subsequent datablocks ...ability encode
                                   the disk-access-data-blocks also as part of leaf-nodes or leaf-blocks.


Opportunity k): Given the kinematic's of data-storage-access on disk-based-mediums ...on virtual-entities-object's as is the case discussed
                in the context.

                         a more-efficient way of representing what is usually done using the UML-use-case's,UML-addressing-problem-statement
                         in a Model-view-controller-architechture ....refer to 'opportunity_storyboard_MVC_Perl_pm_pl.txt'


                Explore the viability of  read through the document opportunity_DWH_BI_product_orchrestrization.txt( in specific as detailed
                ------------------------                                                         envisaged in section opportunity g:))

Opportunity l): bundling storage subsystems 

                       with sufficient cache memory(12 to 16 GB of cache for small system like ds4300 ...a1000,a5000)
                       with sufficient bandwidth between host and the system...fibre-optic-connectivity.

                       rationalize the facts for san based storage fabric's read through opportunity_SAN_fabrics_sharing_devices_quorrum.txt

                                        
               
 
Opportunity m): Scope rationalization
                ----------------------

               PCB hard-disk ??  PCB network-cards ??
               --------------------------------------

               bridging bus-bandwidth ...ease of integrating with ide / scsi  ...virtual-lun / location of physical disk inter/intra
               enclosures ...carrier[media] transport layers from Hosts to enclosures.  


               drawback's of emulation, advantages of emulation ...ability to spread, splinter I/O

                                      ....ability to fine-tune peformance application layer.    


Opportunity n): usage of a storage array ...deriving further to as  envisagable from ....

                backing of system's in a sub-net ...either one-to-one ...on to disk's in a array ...high-end or secure reliability computing,
 
                as also usage of HA ...for bring-up online any designated system online.

                usage of replication / backup ...based re-usable librarie ...for achieving the objective.

                also ...understanding ...deriving from the requirement of buffered ...cache to write ...backup's efficiently to a tape,
                usage of replication ...binaries(differential or incremental on daily or set frequency) to a host connected to with tape device,
                dumping back-up's to the tape (given backup is organized in hirerarcial nature ...refer to the db2backupscript ) ...ability 
                to use  unix (cp ,find directly writing to the device handle) utilities on the tape ...restore or replay 
                ...kind of transaction log ....


                in the above context read through opportunity_generic_api_backup_volumemanagement_storage_mediums.txt,
                                                  opportunity_product_innovation_pluggable_tape_drives.txt
               

                    

Note: The above problem statement having been encountered in various scenarios
        and detailed in various 'Proof of concepts' as mentioned in 
       
       http://uk.geocities.com/ravivenkatus/projects.pdf
       http://ravishankarkv.tripod.com/projects.pdf
        ....apply appropriate
       'use-case' modeling, rationalize and arrive at a workable and feasible 
       solution both commercially and techinically viable.

                  