Monday, 6 January 2014

Building Search Indexes in Websphere Commerce


Websphere commerce has been providing apache solr  embedded on the WAS server as aembedded search engine with the WCS e-commerce solution.
Websphere Commerce Search Engine was started as a part of Feature Pack 2.
  • WebSphere Commerce Version 7 Feature Pack 2
  • Websphere Commerce Version 7 Feature Pack 3  
  Enablement of the Search Feature .
Incase the Feature was not enabled as a part of the Feature Pack installation, the same can be enabled by doing the following.
 enableFeature.bat -DfeatureName=foundation [-DsolrHome=solrhome]



Setting up the solr Search locally .
  1. Navigate to the following directory:
 Run the search index setup utility:
setupSearchIndex.bat -action action-masterCatalogId masterCatalogId [-indextype indextype] [-solrhome solrhome][-setupWebContent true | false] [-indexsubtype indexsubtype]

setupSearchIndex.bat -action action-masterCatalogId masterCatalogId [-indextype indextype] [-solrhome solrhome][-setupWebContent true | false] [-indexsubtype indexsubtype]

setupSearchIndex.bat -masterCatalogId masterCatalogId  ( This will suffice on a Local Developer Machine ).

Sample Logs
=================================================================
WebSphere Commerce Search Index Setup Utility
=================================================================

Utility started at 2013-09-20 12:30:32
Configuration initialization started at 2013-09-20 12:30:32
Configuration initialization finished at 2013-09-20 12:30:33
Database initialization started at 2013-09-20 12:30:33
Database initialization finished at 2013-09-20 12:30:34
Initialization completed in 2.332 seconds.
Build and execute process job started at 2013-09-20 12:30:34


**************************************************************
Started configuring WebSphere Commerce for Solr cores
**************************************************************
Master Catalog Id: 10051
Index Type: CatalogEntry
Languages: [en_US, fr_FR, de_DE, it_IT, es_ES, pt_BR, zh_CN, zh_TW, ru_RU, ko_
, ja_JP, ro_RO, pl_PL]
**************************************************************

--- Started configuring pre-process. ---
--- Finished configuring pre-process. ---

--- Started configuring attribute dictionary. ---
--- Finished configuring attribute dictionary. ---

--- Started configuring classic attribute. ---
--- Finished configuring classic attribute. ---

--- Started configuring delta update. ---
--- Finished configuring delta update. ---

--- Started configuring wc-search.xml. ---
--- Finished configuring wc-search.xml. ---


**************************************************************
Finished configuring WebSphere Commerce for Solr cores
**************************************************************




**************************************************************
Started configuring WebSphere Commerce for Solr cores
**************************************************************
Master Catalog Id: 10051
Index Type: CatalogGroup
Languages: [en_US, fr_FR, de_DE, it_IT, es_ES, pt_BR, zh_CN, zh_TW, ru_RU, ko_
, ja_JP, ro_RO, pl_PL]
**************************************************************

--- Started configuring pre-process. ---
--- Finished configuring pre-process. ---

--- Started configuring delta update. ---
--- Finished configuring delta update. ---

--- Started configuring wc-search.xml. ---
--- Finished configuring wc-search.xml. ---


**************************************************************
Finished configuring WebSphere Commerce for Solr cores
**************************************************************



**************************************************************
Started setup for Solr cores
**************************************************************
Master Catalog Id: 10051
Index Type: CatalogEntry
Languages: [en_US, fr_FR, de_DE, it_IT, es_ES, pt_BR, zh_CN, zh_TW, ru_RU, ko_
, ja_JP, ro_RO, pl_PL]
**************************************************************

--- Started checking directories for setup Solr cores. ---
--- Finished checking directories for setup Solr cores. ---

**************************************************************
The following languages passed directory validation:
- en_US
- fr_FR
- de_DE
- it_IT
- es_ES
- pt_BR
- zh_CN
- zh_TW
- ru_RU
- ko_KR
- ja_JP
- ro_RO
- pl_PL

The following Solr cores will setup:
- MC_10051_CatalogEntry_en_US
- MC_10051_CatalogEntry_Unstructured_en_US
- MC_10051_CatalogEntry_fr_FR
- MC_10051_CatalogEntry_Unstructured_fr_FR
- MC_10051_CatalogEntry_de_DE
- MC_10051_CatalogEntry_Unstructured_de_DE
- MC_10051_CatalogEntry_it_IT
- MC_10051_CatalogEntry_Unstructured_it_IT
- MC_10051_CatalogEntry_es_ES
- MC_10051_CatalogEntry_Unstructured_es_ES
- MC_10051_CatalogEntry_pt_BR
- MC_10051_CatalogEntry_Unstructured_pt_BR
- MC_10051_CatalogEntry_zh_CN
- MC_10051_CatalogEntry_Unstructured_zh_CN
- MC_10051_CatalogEntry_zh_TW
- MC_10051_CatalogEntry_Unstructured_zh_TW
- MC_10051_CatalogEntry_ru_RU
- MC_10051_CatalogEntry_Unstructured_ru_RU
- MC_10051_CatalogEntry_ko_KR
- MC_10051_CatalogEntry_Unstructured_ko_KR
- MC_10051_CatalogEntry_ja_JP
- MC_10051_CatalogEntry_Unstructured_ja_JP
- MC_10051_CatalogEntry_ro_RO
- MC_10051_CatalogEntry_Unstructured_ro_RO
- MC_10051_CatalogEntry_pl_PL
- MC_10051_CatalogEntry_Unstructured_pl_PL

**************************************************************

--- Started copying Solr core files. ---
--- Finished copying Solr core files. ---

--- Started configuring DIH (wc-data-config.xml). ---
--- Finished configuring DIH (wc-data-config.xml). ---

--- Started registering Solr core in solr.xml. ---
--- Finished registering Solr core in solr.xml. ---


**************************************************************
Started setup for Solr cores
**************************************************************
Master Catalog Id: 10051
Index Type: CatalogGroup
Languages: [en_US, fr_FR, de_DE, it_IT, es_ES, pt_BR, zh_CN, zh_TW, ru_RU, ko_
, ja_JP, ro_RO, pl_PL]
**************************************************************

--- Started checking directories for setup Solr cores. ---
--- Finished checking directories for setup Solr cores. ---

**************************************************************
The following languages passed directory validation:
- en_US
- fr_FR
- de_DE
- it_IT
- es_ES
- pt_BR
- zh_CN
- zh_TW
- ru_RU
- ko_KR
- ja_JP
- ro_RO
- pl_PL

The following Solr cores will setup:
- MC_10051_CatalogGroup_en_US
- MC_10051_CatalogGroup_fr_FR
- MC_10051_CatalogGroup_de_DE
- MC_10051_CatalogGroup_it_IT
- MC_10051_CatalogGroup_es_ES
- MC_10051_CatalogGroup_pt_BR
- MC_10051_CatalogGroup_zh_CN
- MC_10051_CatalogGroup_zh_TW
- MC_10051_CatalogGroup_ru_RU
- MC_10051_CatalogGroup_ko_KR
- MC_10051_CatalogGroup_ja_JP
- MC_10051_CatalogGroup_ro_RO
- MC_10051_CatalogGroup_pl_PL

**************************************************************

--- Started copying Solr core files. ---
--- Finished copying Solr core files. ---

--- Started configuring DIH (wc-data-config.xml). ---
--- Finished configuring DIH (wc-data-config.xml). ---

--- Started registering Solr core in solr.xml. ---
--- Finished registering Solr core in solr.xml. ---

Build and execute process job finished at 2013-09-20 12:30:46

Program exiting with exit code: 0.
Search index setup successfully completed without errors.

Utility stopped at 2013-09-20 12:30:46
Total running time is 13.37 seconds.



After you set up the search index structure for a specific master catalog, you must preprocess the search index data.

2. di-preprocess.bat full-path [-fullbuild true | false] [-localename localename] [-onelevel true | false] [-multithread true | false]

eg :

 di-preprocess.bat  C:\WCDE_ENT70\search\pre-processConfig\MC_10051\Cloudscape

The Directories would have formed if the Search index setup had run successfully with exit code 0.


Sample Logs

INFO: F:\WCDE_ENT70\search\pre-processConfig\MC_10051\Cloudscape\CatalogGroup
-dataimport-preprocess-common.xml
Sep 20, 2013 12:40:17 AM com.ibm.commerce.foundation.dataimport.preprocess.Da
mportPreProcessorMain processDataConfig(DataProcessingConfig, String)
INFO: table:
Sep 20, 2013 12:40:17 AM com.ibm.commerce.foundation.dataimport.preprocess.Ab
actDataPreProcessor populateTable(PreparedStatement, Long, Connection)
INFO: The batch is being executed...
Sep 20, 2013 12:40:17 AM com.ibm.commerce.foundation.dataimport.preprocess.Ab
actDataPreProcessor populateTable(PreparedStatement, Long, Connection)
INFO: Batch execution completed.
Sep 20, 2013 12:40:17 AM com.ibm.commerce.foundation.dataimport.preprocess.Da
mportPreProcessorMain processDataConfig
INFO: Data import pre-processing completed in 0.152 seconds for table TI_CATG
EL_3.
Sep 20, 2013 12:40:17 AM com.ibm.commerce.foundation.dataimport.preprocess.Da
mportPreProcessorMain processDataConfig(DataProcessingConfig, String)
INFO: F:\WCDE_ENT70\search\pre-processConfig\MC_10051\Cloudscape\CatalogGroup
-dataimport-preprocess-common.xml
Sep 20, 2013 12:40:17 AM com.ibm.commerce.foundation.dataimport.preprocess.Da
mportPreProcessorMain processDataConfig(DataProcessingConfig, String)
INFO: table:
Sep 20, 2013 12:40:17 AM com.ibm.commerce.foundation.dataimport.preprocess.Ab
actDataPreProcessor populateTable(PreparedStatement, Long, Connection)
INFO: The batch is being executed...
Sep 20, 2013 12:40:17 AM com.ibm.commerce.foundation.dataimport.preprocess.Ab
actDataPreProcessor populateTable(PreparedStatement, Long, Connection)
INFO: Batch execution completed.
Sep 20, 2013 12:40:17 AM com.ibm.commerce.foundation.dataimport.preprocess.Da
mportPreProcessorMain processDataConfig
INFO: Data import pre-processing completed in 0.171 seconds for table TI_DPGR
_3.
Sep 20, 2013 12:40:17 AM com.ibm.commerce.foundation.dataimport.preprocess.Da
mportPreProcessorMain processDataConfig(DataProcessingConfig, String)
INFO: F:\WCDE_ENT70\search\pre-processConfig\MC_10051\Cloudscape\CatalogGroup
-dataimport-preprocess-common.xml
Sep 20, 2013 12:40:17 AM com.ibm.commerce.foundation.dataimport.preprocess.Da
mportPreProcessorMain processDataConfig(DataProcessingConfig, String)
INFO: table:
Sep 20, 2013 12:40:20 AM com.ibm.commerce.foundation.dataimport.preprocess.Da
mportPreProcessorMain processDataConfig
INFO: Data import pre-processing completed in 2.89 seconds for table TI_CGSEO
_3_#lang_tag#.
Sep 20, 2013 12:40:20 AM com.ibm.commerce.foundation.dataimport.preprocess.Da
mportPreProcessorMain logExitCode
INFO:
The program exiting with exit code: 0.
Data import pre-processing completed successfully with no errors.

Sep 20, 2013 12:40:20 AM com.ibm.commerce.foundation.dataimport.preprocess.Da
mportPreProcessorMain logEndDateAndTime
INFO: Data import pre-processing ended:Fri Sep 20 00:40:20 IST 2013
Sep 20, 2013 12:40:20 AM com.ibm.commerce.foundation.dataimport.preprocess.Da
mportPreProcessorMain logEndDateAndTime
INFO: Data import pre-processing completed in 56.491 seconds.


Verify that the Program executed successfully.

Also, inspect the following file for errors:
To get more logging information, update the logging level from INFO to FINEST in the following file: 
# Default global logging level, INFO
.level=FINEST
You can also increase the log file size and the number of log files. For example:
# Limiting size of output file in bytes:
java.util.logging.FileHandler.limit=50000000

# Number of output files to cycle through
java.util.logging.FileHandler.count=2

Well now for the last step .

After preprocessing the search index data, you must build the search index.

Start the server.

 navigate to bin folder.
run the following.

di-buildindex.bat -instance instance_name -masterCatalogId masterCatalogId [-indextype indextype] -dbuser dbuser -dbuserpwd dbuserpwd [-fullbuild true | false] [-statusInterval statusInterval] [-localename localename] [-indexSubType indexSubType] [-basePath basePath] [-storeId storeId] [-webcontentDelete true | false] [-solrConnTimeout solrConnTimeout] [-soTimeout soTimeout] [-workspace workspaceId] [-force true | false


eg:  di-buildindex.bat -masterCatalogId 10051 -basePath C:\WCDE_ENT70\workspace\StaticContent\WebContent\Madisons\StaticContent\en_US

In this case i have a seperate project StaticContent.

Posted by at 8 comments:
Subscribe to: Posts (Atom)