FileNet Content Services Concepts - Part 5

Replication Services Overview

Replication Services gives your documents greater availability across your enterprise, automatically and without compromising document integrity. Instead of manually creating and maintaining the necessary user accounts and documents on separate libraries, replication automatically copies the documents and related security to another library system.

Replication Services provides a way to make documents and items in one library system available to other library systems. Replication Services automatically copies documents, items, and their security to additional specified library systems. As documents are checked out, updated, and checked in again, the changes are synchronized across the other libraries.

When a document is replicated, both the file content and the property values for standard properties and existing custom properties of the document are copied into the library systems participating in Replication Services. Users can quickly locate and access a replicated document anywhere in any replicated library. All instances of replicated documents or folders in any library system are periodically synchronized with any alterations to the original document or folder.

However, the synchronization is not immediate. That is, Replication is an asynchronous store-and-forward system. Changes are logged into a queue for processing, and then propagated to the replica libraries. The processing may take place almost immediately, or it may take place only during certain hours of the day, depending on your configuration parameters, the number and type of changes requested, the network load, and the availability of the replica libraries. For example, if a replica library is offline for maintenance (or a power failure), the update cannot be completed until the library is available again.

A replicated document's current status is reflected on all the participating library systems: if checked out, a document will be marked as checked out on all of these library systems. When a new version is checked in, that new version is replicated to all the participating library systems. Replication Services runs independently of the other library system services. Thus, Replication Services can continue running while other library system services are stopped and restarted and can automatically resynchronize with these library system services when they restart. Using Replication Services can have a profound impact on your library system configurations and the hardware and software resources required. Take time to carefully plan your system before you implement replication in your network.

Additional point: You must copy Document Classes and Controlled Vocabulary Lists (CVLs) from one library to another using the import and export tools available in CS Explorer.

FileNet Content Services Concepts - Part 4

Deleting Sessions

In a library system, an active session is the open session that a user is currently working in. There are actually two sessions involved when a user is logged in to a library system via a Content Services client application, such as IDM Desktop or CS Web Admin environment. One is the database connection session, and the second is the Content Services library session.

When a CS client application exits or is killed, the corresponding database connection session is deleted from the database. You can also delete all the database connections at once by stopping and restarting the database. If a database connection session is deleted, then any corresponding active CS library session is said to be an orphan session.

CS library system sessions are tracked by CS Explorer but cannot be removed by it. Neither stopping and restarting the database or CS services will delete an orphan session. Nor will rebooting a workstation where a CS client application is running. To remove these orphan CS library sessions, you can invoke DSSTOP on the server where the library system is resident. Alternatively, you can simply allow the daemon running on the database custodial server to periodically check for suspended sessions. The daemon will remove any orphaned sessions that are more than two days old. To find out what sessions are open in a particular library system and to see whether these sessions are active or suspended, you can check the status of the Sessions object using CS Explorer.

Resetting the Session ID

The library system maintains a unique session ID for each library system session. The session ID is incremented each time a user opens a new session. Over time, it is possible to exceed the limit of approximately 2 billion (231) session IDs.

To avoid an overflow condition, we recommend that you periodically check the value of the Session Number  property in a Session object (the value of this property is the session ID). When the value of the Session Number property approaches 2 billion, you should reset the session ID to 0.

To reset the session ID:

1. Stop the library system and database services.

2. Restart the database services.

3. Using your SQL query tool (SQL Server Query Analyzer for SQL Server or sqlplus for Oracle), reset the session ID to 0.

• For SQL Server enter:

use system_name

go

delete from session

go

update numid set se_id_num=0

go

• For Oracle enter:

delete from system_name.session;

update system_name.numid set se_id_num=0

4. Restart the library system services

FileNet Content Services Concepts - Part 3

Secure Document Delete

With the Secure Document Delete feature you can scrub your sensitive document files to prevent any possibility of recovery via disk recovery tools. Scrubbing means to overwrite its contents with a byte pattern before actually deleting the file.

Scrubbing means that the file's entire content is overwritten by a byte pattern before the file is actually deleted. Secure Document Delete does not delete files from client machines. To ensure secure delete functionality on servers containing library system components, do not install client interfaces (such as IDM Desktop) on those servers.

Neither Replication Services nor Web Services scrub files during add, checkin, and checkout operations. For simple delete operations as may be desired during a “purge” process however, Secure Document Delete is applied on all servers that have this feature turned on.

You can configure the Secure Document Delete feature for each Storage Manager by selecting one of the options described below.

Level	Description
No Secure Deletes	Ordinary delete. Files are not overwritten before being deleted.
One Scrub	Files are overwritten with zeroes once before being deleted. This level corresponds to the Clear definition of the DOD 5220.22-M specification.
Three Scrubs	Files are overwritten once with an arbitrary character, again by the character's complement, and finally by a random character before being deleted. This level corresponds to the Purge or Sanitize definition of the DOD 5220.22-M specification.

The initial and additional (remote) Storage Managers of a library system are each separately configurable with respect to secure deletes. Whether an item (document) is securely deleted depends on the secure-delete setting of all the Storage Managers that govern the storage repositories where the versions of the item are stored.

When an item is added to a library, it is given a Storage Category, which is associated with the storage repository where the versions of the item will be stored. As one storage repository fills up, the system administrator will assign the Storage Category to another storage repository. Thus, over time, different versions of the same item may end up in different storage repositories.

To ensure that an item is securely deleted, make sure it is given a Storage Category all of whose associated storage repositories will only be on servers where the Storage Manager is configured for secure deletes. Beware that if even one of the storage repositories associated with the Storage Category is on a server whose Storage Manager is not configured for secure deletes, then any versions of the item that are stored on that server will not be securely deleted.

Secure Document Delete Limitations

On UNIX platforms, files larger than 2 Gbytes will not be overwritten and must be scrubbed using special tools. On Windows platforms, files will be scrubbed securely up to the full Windows 64-bit limit.

Secure Document Delete does not scrub property data in the database; this remains the job of the database administrator and the database itself. Microsoft SQL Server and Oracle do not scrub database data during row or table deletion.

You cannot obtain Secure Document Delete functionality on compressed disks or files. Do not use defragmentation tools on the disk containing the stored document files. Such tools move the files on the disk without pre-scrubbing them, making secure deletes unsupportable.

Some database metadata (such as file names, Item IDs, Version IDs) are not scrubbed in subtle places. For example, when a search query is performed, even on the server, a temp file is used to store the results. When the query ends, the temp file is deleted (but not securely). Database metadata is generally insecure; even Microsoft SQL Server and Oracle databases cannot do secure metadata deletes.

The Windows pagefile and UNIX swap files may include memory images that need to be paged to disk, and the memory image may include the files you are copying or writing. The Secure Document Delete functionality cannot scrub these files.

Reformatting a hard disk may not overwrite data on the disk. If you reformat a disk containing storage repositories, you should use other tools to scrub the disk clean of file data.

The uninstall command (dsuninst.exe) does not scrub storage or index repository files.  Activities such as the actions of the executable files that perform during installs and upgrades, the use of tools, and utilizing configuration code, do not carry out secure deletes. For example when DLLs are copied to a temp directory, they are not scrubbed when they are removed. Secure Document Delete does not work on Hierarchical Storage Manager extended drives. HSM controls access to files on media and cannot guarantee scrubbing in the process of a move or purge

FileNet Content Services Concepts - Part 2

Library System Security

One of the powerful feature of the library system is the security it provides for your important information. Each time you or any other user adds a document to the library system, you determine the document's descriptive properties and its associated version files and properties that users and groups will be able to access. You provide this information using the document's Access List property. For further security, there is also an Access List property  for all users and groups. Three objects commonly referenced by users and administrators contain an access list: the User object, the Group object, and the Item object. However, access rights to entities without access lists (such as versions) are passed down through “parent” objects (such as the Item object). So, the library system provides a simple way to set up and maintain access control, yet allows this security to be as controlled or permissive as necessary. There are five levels of access rights in a library system:

Access Right	Privileges granted
None	No access. If no other access level is stated in the access list, access rights of None are assumed. An access level of None can also be explicitly stated in an access list.
Viewer	Generally, the ability to view the object properties or to make copies of the associated versions.
Author	(Applies to documents and versions only) Viewer access rights plus the ability to checkout ,check in and copy associated versions and modify property values for the version. In addition, you may be allowed to modify designated custom property values for the document.
Owner	Author access rights plus the ability to delete documents, modify security and modify most properties.
Admin	Owner access rights plus the ability to modify all property values. Active members of the Administrators group are automatically assigned Admin access rights to all properties, even though their names do not appear in any access lists. Users who are not members of the Administrators group can be explicitly assigned an Admin access level to the properties associated with particular objects.

DIAL – Default item access list

Access List Defaults

To ensure that each access list contains initial entries, a library system provides some defaults in the User, Group and Item object. In User and Group object access lists, these defaults are standard entries that are always added, as shown in the following:

Name                           Type                Access Level

(Added By User)         User                 Admin

(User's Name)              User                 Owner

General Users              Group              Viewer

The Added By User value is the user who added the object to the library system. This user and any user with Owner or Admin access rights can modify the values after the User or Group object has been created. The Administrators group always has Admin access rights, even though these rights are not displayed in the access list.

Default Item Access Lists

Another way a library system can help users control access to their files is by inserting default entries in document access lists so that the user does not have to provide the same set of entries each time he or she adds a document. You can specify a set of default item access list entries for each user in his or her User object. Then, each time that user adds a document to the library system from any user interface, the access list of the Item object is filled in with those specified defaults. Of course, to cover special cases, users can always change the access list of any documents they add, but they do not have to start from scratch with each document. In the same way that you add entries to the default item access list in the User object, you can also specify them in Group objects and the System object.

Thus, the access lists of documents do not necessarily have a standard set of default entries. The library system does add default entries to the document's access list as the document is added, but to determine these defaults, the library system will check for entries in the Default Item Access List properties in the following objects and use the first such list with any entries:

1. The User object

2. The Group object for the user's active group

3. The System object

To understand how default item access lists work, consider the following example. At the law firm of Hunter and Bowers, senior lawyer Sarah Black is using IDM Desktop to add an item to a library system. She knows that her system administrator has given her active group (Attorneys) the default item access list (shown below), which will be applied to all documents added by the group's members.

Name                           Type                Access Level

Attorneys                     Group              Author

Managers                     Group              Author

Paralegals                     Group              Viewer

The system administrator has left the Default Item Access List property blank in Sarah's User object because Sarah  always wants the default access list for her active group to be applied. (Likewise, whenever Sarah changes her active group, the library system will apply the default item access list of her new active group.) She also realizes that since the default access list at her active group level does not mention her name specifically, she will receive the program default access rights for any user who adds a document and Owner access rights to the document that she is adding to the library system. And with Owner access rights to the new document, she can modify the entries in the document's access list at a later time if necessary.

FileNet Content Services Concepts - Part 1

The Library System

Content Services (also called the library system) provides server-based enterprise content management (ECM) that can be accessed via a client interface using applications such as FileNet IDM Desktop or FileNet Web Services. An enterprise, especially if it's spread across various sites, may have multiple library systems, depending upon sizing and load balancing considerations.

A typical Content Services installation comprises the following components:

• One or more library systems.

• One or more administrative clients.

• One or more ECM clients (such as IDM Desktop).

• One or more application solutions (such as FileNet Web Services for intranet/internet access).

As you can see in the following illustration, each library system has a set of sub-components that you install and configure to provide the services for your various client-based users.

Each library system acts as an information storehouse.

·         The Property Manager stores and manages object properties (users, groups, system, and custom properties) and metadata describing your catalogued documents.

·         The Storage Manager stores the actual documents and process requests.

·         The Content Search Manager provides document content-based indexing and searching capabilities.

·         CS Replication Services, which allows you to replicate documents across multiple library systems.

You have the option to locate your database on a standalone property server. In this configuration, the initial Storage Manager, Content Search Manager, and Replication Services would be located not on the property server, but on the initial storage server.

Protected Items

When you add a document to a library system, you usually add it as a protected item. It is called “protected” because the library system physically stores the document, thereby protecting it from unauthorized access, inadvertent removal, and more. Protected items can be documents that were created by any application (for example, word-processing documents, financial spreadsheets, images, audio, tables, and so on).

Unprotected Items

A library system also lets you control and manage unprotected items (external documents), which are items that are not physically stored in the library system but are tracked for easy management. This capability is especially useful for items that are stored in special locations, yet still require version tracking or controlled access. Some examples are: magnetic tapes, printed technical manuals, printed engineering drawings, and software source code listings. Although the unprotected items added at most sites represent these kinds of physical objects, unprotected items can also be electronic files that you do not want to store in the library system.

FileNet Content Services Features

Network-wide Search and Retrieval

FileNet Content Services (CS) systems allow documents to be identified using plain language titles and property names, not cumbersome file names. End users can search for documents based on their properties and/or their content. Regardless of the user's location or the file's location, complex path commands with coded document names are never required. Documents are checked out of and checked in to the FileNet CS system for updates and revisions. One simple end user command transparently saves, tracks, and archives any document.

Logical, Not Physical, Storage Pointers

Unlike conventional document management packages that rely on physical file locations, a FileNet CS system does not require users to have any knowledge of where documents are actually stored. In fact, users are shielded from that knowledge, providing an added layer of security for the documents and the network. Document management systems that use physical pointers require constant supervision and cumbersome revision as networks grow and change.Moving files, adding users adding or changing directories means updating every single workstation individually. With a FileNet CS system, such system changes can be handled centrally by the network administrator.

Object-Oriented Approach

A FileNet CS system encapsulates all types of files and objects. Each object is described by properties such as description, security, and administration. Actions are performed against objects using these properties as variables. In this way, an ordinary file becomes an “intelligent” document because, regardless of how or where it is used, the document retains its properties and always knows who is authorized to view it or make edits, where it is to be stored on the network, and how it is to be administered and archived.

Version Control

When a user opens a document for editing, it is designated as checked out. The user is always provided a copy of the particular version of the document, and the original copy of that version remains protected. Other users who try to access a document version that is checked out are notified that it is checked out and by whom. While the document is checked out and being modified, users can view the latest checked-in version of the document. When the user is finished working with a checked out version, he or she can check it in. The FileNet CS system retains the old version and automatically saves the checked in document as a new version. Once checked in, the old and new versions of the document become available for check out to other users. When users access the document, they are free to check out the most current version or go back to older versions as needed.

Multilevel Security

An access control list is created for each document added to a FileNet CS system. This access control is part of the document's properties and is independent of the document's location on the network. In this way, authorized users have access to the information they need, no matter where they or the documents are physically located. Yet all stored documents are well protected, because only the documents that each user is authorized to see will be displayed as the result of a search.

A FileNet CS system controls who can create new versions of any given document, regardless of data type, application used, or storage location. Security levels determine access rights such as Owner, Author, Viewer, or None. The only access to documents is through the FileNet CS system. File names are encrypted and users are prevented from bypassing the FileNet CS system to access files via the network operating system. CS administrators should not share the directories where FileNet file are located.

ECM Administration Tools

FileNet CS systems include a Windows® based administrative tool. ECM administration tasks such as adding new servers, designating new storage locations, or adding new users and groups can be performed at any time, no matter how many users are online. In addition, a web browser-based version of the administrative tool that includes a subset of the windows-based administration tool is available for administrators who want mobile access to library system properties and the ability to start/stop CS servers remotely.

Flexible Open Technology

CS is designed to work with virtually every combination of user interface, application, and  hardware/software. Creation and maintenance of general office documents (including enterprise, departmental, workgroup, and personal documents), legal document management, email, as well as document imaging may all be handled through a common repository. This means that no matter how many platforms or environments are used across the network, all enterprise applications are interoperable. Additionally, any enterprise can take advantage of FileNet IDM Desktop (and, if necessary, CS API programming functions) to build custom enterprise applications as future needs emerge.

Load Balancing and Scalability of Distributed Services

No matter how an organization's workload grows and changes across departments or divisions, the  FileNet CS system's queue management and load balancing features ensure a consistent working environment for end users.

Documentum Foundation Classes 6

DFC (Documentum Foundation Classes 6.0)

DFC is a key part of the Documentum software platform. While the main user of DFC is other Documentum software, you can use DFC in any of the following ways:

• Access Documentum functionality from within one of your company’s enterprise applications.

For example, your corporate purchasing application can retrieve a contract from your Documentum system.

• Customize or extend products such as Webtop.

For example, you can modify Webtop functionality to implement one of your company’s business rules.

• Write a method or procedure for Content Server to execute as part of a workflow or document

lifecycle.

For example, the procedure that runs when you promote an XML document might apply a transformation to it and start a workflow to subject the transformed document to a predefined business process.

You can view Documentum functionality as having the following elements:

Repositories One or more places where you keep the content and associated metadata of your organization’s information. The metadata resides in a relational database, and the content resides in various storage elements.

Content Server

Software that manages, protects, and imposes an object oriented structure on the information in repositories. It provides tools for managing the lifecycles of that information and automating processes for manipulating it.

Client programs

Software that provides interfaces between Content Server and end users. The most common clients run on application servers (for example, Webtop).

End Users People who control, contribute, or use your organization’s information. They use a browser to access client programs running on application servers.

Documentum Foundation Classes (DFC) lies between Content Server and clients. Documentum Foundation Services are the primary client interface to the Documentum platform. Documentum Foundation Classes are used for server‑side business logic and customization.

DFC is Java based. As a result, client programs that are Java based can interface directly with DFC.When application developers use DFC, it is usually within the customization model of a Documentum client, though you can also use DFC to develop the methods associated with Content Server functionality, such as document lifecycles.

In the Java application server environment, Documentum client software rests on the foundation provided by the Web Development Kit (WDK). This client has a customization model that allows you to modify the user interface and also implement some business logic. However, the principal tool for

adding custom business logic to a Documentum system is to use the Business Object Framework (BOF).

BOF enables you to implement business rules and patterns as reusable elements, called modules. The most important modules for application developers are type based objects (TBOs), service based

objects (SBOs), and Aspects. Aspect modules are similar to TBOs, but enable you to attach properties and behaviour on an instance‑by‑instance basis, independent of the target object’s type.

BOF makes it possible to extend some of DFC’s implementation classes. As a result, you can introduce new functionality in such a way that existing programs begin immediately to reflect changes you make to the underlying business logic.

The Documentum Content Server Fundamentals manual provides a conceptual explanation of the capabilities of Content Server. DFC provides a framework for accessing those capabilities. Using DFC and BOF makes your code much more likely to survive future architectural changes to theDocumentum system.

Where Is DFC?

DFC runs on a Java virtual machine (JVM), which can be on:

• The machine that runs Content Server.

For example, to be called from a method as part of a workflow or document lifecycle.

• A middle‑tier system.

For example, on an application server to support WDK or to execute server methods.

For client machines, Documentum 6 now provides Documentum Foundation Services (DFS) as the primary support for applications communicating with the Documentum platform.

Acronyms

DFC      Documentum Foundation Classes

TBO     Type based object

SBO     Service based object

WDK     Web development toolkit

BOF     Business object framework

Microsoft Dynamic Langugage Runtime (DLR)

The Dynamic Language Runtime (DLR) is a set of libraries built on the CLR to support dynamic language implementations on .NET. A key value proposition of the .NET CLR is supporting multiple languages and allowing them to inter operate with each other. Dynamic languages have become very popular in the last several years. Customers want to use their favorite dynamic language and have great .NET interoperability for building applications and providing scripting for applications. The DLR makes it very easy to develop dynamic languages on .NET.
The DLR provides three key components:

1.  language implementation services with language interoperability model
2.  dynamic language runtime services with fast dynamic dispatch and library support
3.  common hosting APIs across languages

The key goals of the DLR are making it easy to

1.  port dynamic languages to .NET
2.  add dynamic features to your existing language
3.  author libraries whose objects support dynamic operations
4.  employ dynamic languages in your applications and frameworks.

DLR hosting APIs
The DLR Hosting API is a programming interface that allows one language’s code to execute in another
language. It helps in using a Dynamic Language’s Code in a Static Language.

Dynamic Languages - Outside .net family of languages. For example: Ruby or Python
Static Languages - .NET languages like VB.net, C#.net etc.
For more information please visit http://dlr.codeplex.com/

K2 BlackPoint

K2 BlackPoint is used to create new workflow using Microsoft Sharepoint and Microsoft Infopath.

It has two designers a). K2 Studio and b). K2 Web Designer

Benefits:
a). Visual tools to create worflow.
b). No coding required.
c). Tools that can be used by non developer.

For more information visit http://www.k2.com/en/blackpoint.aspx

Generate barcodes using barcode4j

Barcode4J is a barcodes generator written in Java. It's free, available under the Apache License, version 2.0.It supports generating 1D and 2D barcodes of various types.

a) Implementations

* 1D barcode implementations
          o Interleaved 2 of 5
          o Code 39
          o Code 128
          o Codabar
          o POSTNET & more
 * 2D barcode implementations :
          o PDF 417 (ISO/IEC 15438:2001(E))
          o DataMatrix (ISO/IEC 16022:2000(E))

b) Support for multiple formats:

It supports various file formats viz. SVG, EPS (Encapsulated PostScript), Bitmap images (such as PNG or JPEG),  Java2D (AWT), Text (for testing and debugging only).
http://barcode4j.sourceforge.net/2.0/output-formats.html

c)Barcode Servlet with support for SVG, EPS and bitmap output

d) Command-line interface
barcode4j has command line support so you can generate barcode by executing a command on command line.

e) Plug-ins/extensions for third-party products:
          o Apache Xalan: SVG-generating XSLT extension
          o SAXON XSLT Processor : SVG-generating XSLT extension
          o Apache FOP: support as fo:instream-foreign-object

For more information go to http://barcode4j.sourceforge.net/

IBM Classification Module and Content Extractor

IBM® Classification Module

IBM® Classification Module helps organize unstructured content by analyzing the full text of documents and e-mails and applying rules that automate classification decisions.

IBM Classification Module reduces the burden of manual decision making that is done by employees by accurately and automatically organizing information.

It is embedded with natural language processing and semantic analysis capabilities,

CLASSIFICATION WORKBENCH

An application that is used to create and analyze knowledge bases and decision plans. With Classification Workbench, you can also evaluate system performance by importing analysis data and viewing reports and graphical diagnostics.

KNOWLEDGE BASE

A single file encapsulating data that is required by the Classification Module for accurate content-based classification

DECISION PLAN

A collection of rules built in Classification Workbench that determine how the Classification Module classifies content items such as documents or e-mails. Each rule consists of one trigger and one or more actions.

CLASSIFICATION CENTER

A Web application provided with the IBM FileNet P8 integration that is used to manage the classification processes. You can use the Classification Center to determine the content to be classified, specify classification options (such as the decision plan to use and various runtime preferences), monitor classification activity, and view the classification results

 

Content Extractor

A command-line tool provided with the IBM FileNet P8 integration that is used to extract the content from an IBM FileNet P8 object store. You can import the extracted content into Classification Workbench and use it to train a knowledge base or provide test data for a decision plan.

It uses a properties file where option for what to be extracted is specified and extracts the document in XML format.

IBM WebSphere Transformation Extender (WTX)

Its a dedicated data transformation engine for app integration. It’s being used in organizations around the world. A transformation engine accepts FILE docs or electronic messages as inputs, parsed them using data dictionary like definitions & transforms them into one or more o/p messages.

It can convert all most any formatted content and does this without any coding/scripting by developer. WTX is a family of products and has editions tailored to ESB, BPM and B2B products.

How does it connect to existing applications?

WTX has local file system support by default. This helps for testing transformations thoroughly before moving to the connectivity phase.

How does WTX aid application reuse?

For example Industry Standards, they get updated usually one/twice a year. As organizations are required to follow the standards so the rules are coded in to the apps. You can end up in spending so much time and money on maintenance and not on new projects. WTX protects apps from change and allow them to participate in new projects.