Monday, 6 January 2014

Building Search Indexes in Websphere Commerce


Websphere commerce has been providing apache solr  embedded on the WAS server as aembedded search engine with the WCS e-commerce solution.
Websphere Commerce Search Engine was started as a part of Feature Pack 2.
  • WebSphere Commerce Version 7 Feature Pack 2
  • Websphere Commerce Version 7 Feature Pack 3  
  Enablement of the Search Feature .
Incase the Feature was not enabled as a part of the Feature Pack installation, the same can be enabled by doing the following.
 enableFeature.bat -DfeatureName=foundation [-DsolrHome=solrhome]



Setting up the solr Search locally .
  1. Navigate to the following directory:
 Run the search index setup utility:
setupSearchIndex.bat -action action-masterCatalogId masterCatalogId [-indextype indextype] [-solrhome solrhome][-setupWebContent true | false] [-indexsubtype indexsubtype]

setupSearchIndex.bat -action action-masterCatalogId masterCatalogId [-indextype indextype] [-solrhome solrhome][-setupWebContent true | false] [-indexsubtype indexsubtype]

setupSearchIndex.bat -masterCatalogId masterCatalogId  ( This will suffice on a Local Developer Machine ).

Sample Logs
=================================================================
WebSphere Commerce Search Index Setup Utility
=================================================================

Utility started at 2013-09-20 12:30:32
Configuration initialization started at 2013-09-20 12:30:32
Configuration initialization finished at 2013-09-20 12:30:33
Database initialization started at 2013-09-20 12:30:33
Database initialization finished at 2013-09-20 12:30:34
Initialization completed in 2.332 seconds.
Build and execute process job started at 2013-09-20 12:30:34


**************************************************************
Started configuring WebSphere Commerce for Solr cores
**************************************************************
Master Catalog Id: 10051
Index Type: CatalogEntry
Languages: [en_US, fr_FR, de_DE, it_IT, es_ES, pt_BR, zh_CN, zh_TW, ru_RU, ko_
, ja_JP, ro_RO, pl_PL]
**************************************************************

--- Started configuring pre-process. ---
--- Finished configuring pre-process. ---

--- Started configuring attribute dictionary. ---
--- Finished configuring attribute dictionary. ---

--- Started configuring classic attribute. ---
--- Finished configuring classic attribute. ---

--- Started configuring delta update. ---
--- Finished configuring delta update. ---

--- Started configuring wc-search.xml. ---
--- Finished configuring wc-search.xml. ---


**************************************************************
Finished configuring WebSphere Commerce for Solr cores
**************************************************************




**************************************************************
Started configuring WebSphere Commerce for Solr cores
**************************************************************
Master Catalog Id: 10051
Index Type: CatalogGroup
Languages: [en_US, fr_FR, de_DE, it_IT, es_ES, pt_BR, zh_CN, zh_TW, ru_RU, ko_
, ja_JP, ro_RO, pl_PL]
**************************************************************

--- Started configuring pre-process. ---
--- Finished configuring pre-process. ---

--- Started configuring delta update. ---
--- Finished configuring delta update. ---

--- Started configuring wc-search.xml. ---
--- Finished configuring wc-search.xml. ---


**************************************************************
Finished configuring WebSphere Commerce for Solr cores
**************************************************************



**************************************************************
Started setup for Solr cores
**************************************************************
Master Catalog Id: 10051
Index Type: CatalogEntry
Languages: [en_US, fr_FR, de_DE, it_IT, es_ES, pt_BR, zh_CN, zh_TW, ru_RU, ko_
, ja_JP, ro_RO, pl_PL]
**************************************************************

--- Started checking directories for setup Solr cores. ---
--- Finished checking directories for setup Solr cores. ---

**************************************************************
The following languages passed directory validation:
- en_US
- fr_FR
- de_DE
- it_IT
- es_ES
- pt_BR
- zh_CN
- zh_TW
- ru_RU
- ko_KR
- ja_JP
- ro_RO
- pl_PL

The following Solr cores will setup:
- MC_10051_CatalogEntry_en_US
- MC_10051_CatalogEntry_Unstructured_en_US
- MC_10051_CatalogEntry_fr_FR
- MC_10051_CatalogEntry_Unstructured_fr_FR
- MC_10051_CatalogEntry_de_DE
- MC_10051_CatalogEntry_Unstructured_de_DE
- MC_10051_CatalogEntry_it_IT
- MC_10051_CatalogEntry_Unstructured_it_IT
- MC_10051_CatalogEntry_es_ES
- MC_10051_CatalogEntry_Unstructured_es_ES
- MC_10051_CatalogEntry_pt_BR
- MC_10051_CatalogEntry_Unstructured_pt_BR
- MC_10051_CatalogEntry_zh_CN
- MC_10051_CatalogEntry_Unstructured_zh_CN
- MC_10051_CatalogEntry_zh_TW
- MC_10051_CatalogEntry_Unstructured_zh_TW
- MC_10051_CatalogEntry_ru_RU
- MC_10051_CatalogEntry_Unstructured_ru_RU
- MC_10051_CatalogEntry_ko_KR
- MC_10051_CatalogEntry_Unstructured_ko_KR
- MC_10051_CatalogEntry_ja_JP
- MC_10051_CatalogEntry_Unstructured_ja_JP
- MC_10051_CatalogEntry_ro_RO
- MC_10051_CatalogEntry_Unstructured_ro_RO
- MC_10051_CatalogEntry_pl_PL
- MC_10051_CatalogEntry_Unstructured_pl_PL

**************************************************************

--- Started copying Solr core files. ---
--- Finished copying Solr core files. ---

--- Started configuring DIH (wc-data-config.xml). ---
--- Finished configuring DIH (wc-data-config.xml). ---

--- Started registering Solr core in solr.xml. ---
--- Finished registering Solr core in solr.xml. ---


**************************************************************
Started setup for Solr cores
**************************************************************
Master Catalog Id: 10051
Index Type: CatalogGroup
Languages: [en_US, fr_FR, de_DE, it_IT, es_ES, pt_BR, zh_CN, zh_TW, ru_RU, ko_
, ja_JP, ro_RO, pl_PL]
**************************************************************

--- Started checking directories for setup Solr cores. ---
--- Finished checking directories for setup Solr cores. ---

**************************************************************
The following languages passed directory validation:
- en_US
- fr_FR
- de_DE
- it_IT
- es_ES
- pt_BR
- zh_CN
- zh_TW
- ru_RU
- ko_KR
- ja_JP
- ro_RO
- pl_PL

The following Solr cores will setup:
- MC_10051_CatalogGroup_en_US
- MC_10051_CatalogGroup_fr_FR
- MC_10051_CatalogGroup_de_DE
- MC_10051_CatalogGroup_it_IT
- MC_10051_CatalogGroup_es_ES
- MC_10051_CatalogGroup_pt_BR
- MC_10051_CatalogGroup_zh_CN
- MC_10051_CatalogGroup_zh_TW
- MC_10051_CatalogGroup_ru_RU
- MC_10051_CatalogGroup_ko_KR
- MC_10051_CatalogGroup_ja_JP
- MC_10051_CatalogGroup_ro_RO
- MC_10051_CatalogGroup_pl_PL

**************************************************************

--- Started copying Solr core files. ---
--- Finished copying Solr core files. ---

--- Started configuring DIH (wc-data-config.xml). ---
--- Finished configuring DIH (wc-data-config.xml). ---

--- Started registering Solr core in solr.xml. ---
--- Finished registering Solr core in solr.xml. ---

Build and execute process job finished at 2013-09-20 12:30:46

Program exiting with exit code: 0.
Search index setup successfully completed without errors.

Utility stopped at 2013-09-20 12:30:46
Total running time is 13.37 seconds.



After you set up the search index structure for a specific master catalog, you must preprocess the search index data.

2. di-preprocess.bat full-path [-fullbuild true | false] [-localename localename] [-onelevel true | false] [-multithread true | false]

eg :

 di-preprocess.bat  C:\WCDE_ENT70\search\pre-processConfig\MC_10051\Cloudscape

The Directories would have formed if the Search index setup had run successfully with exit code 0.


Sample Logs

INFO: F:\WCDE_ENT70\search\pre-processConfig\MC_10051\Cloudscape\CatalogGroup
-dataimport-preprocess-common.xml
Sep 20, 2013 12:40:17 AM com.ibm.commerce.foundation.dataimport.preprocess.Da
mportPreProcessorMain processDataConfig(DataProcessingConfig, String)
INFO: table:
Sep 20, 2013 12:40:17 AM com.ibm.commerce.foundation.dataimport.preprocess.Ab
actDataPreProcessor populateTable(PreparedStatement, Long, Connection)
INFO: The batch is being executed...
Sep 20, 2013 12:40:17 AM com.ibm.commerce.foundation.dataimport.preprocess.Ab
actDataPreProcessor populateTable(PreparedStatement, Long, Connection)
INFO: Batch execution completed.
Sep 20, 2013 12:40:17 AM com.ibm.commerce.foundation.dataimport.preprocess.Da
mportPreProcessorMain processDataConfig
INFO: Data import pre-processing completed in 0.152 seconds for table TI_CATG
EL_3.
Sep 20, 2013 12:40:17 AM com.ibm.commerce.foundation.dataimport.preprocess.Da
mportPreProcessorMain processDataConfig(DataProcessingConfig, String)
INFO: F:\WCDE_ENT70\search\pre-processConfig\MC_10051\Cloudscape\CatalogGroup
-dataimport-preprocess-common.xml
Sep 20, 2013 12:40:17 AM com.ibm.commerce.foundation.dataimport.preprocess.Da
mportPreProcessorMain processDataConfig(DataProcessingConfig, String)
INFO: table:
Sep 20, 2013 12:40:17 AM com.ibm.commerce.foundation.dataimport.preprocess.Ab
actDataPreProcessor populateTable(PreparedStatement, Long, Connection)
INFO: The batch is being executed...
Sep 20, 2013 12:40:17 AM com.ibm.commerce.foundation.dataimport.preprocess.Ab
actDataPreProcessor populateTable(PreparedStatement, Long, Connection)
INFO: Batch execution completed.
Sep 20, 2013 12:40:17 AM com.ibm.commerce.foundation.dataimport.preprocess.Da
mportPreProcessorMain processDataConfig
INFO: Data import pre-processing completed in 0.171 seconds for table TI_DPGR
_3.
Sep 20, 2013 12:40:17 AM com.ibm.commerce.foundation.dataimport.preprocess.Da
mportPreProcessorMain processDataConfig(DataProcessingConfig, String)
INFO: F:\WCDE_ENT70\search\pre-processConfig\MC_10051\Cloudscape\CatalogGroup
-dataimport-preprocess-common.xml
Sep 20, 2013 12:40:17 AM com.ibm.commerce.foundation.dataimport.preprocess.Da
mportPreProcessorMain processDataConfig(DataProcessingConfig, String)
INFO: table:
Sep 20, 2013 12:40:20 AM com.ibm.commerce.foundation.dataimport.preprocess.Da
mportPreProcessorMain processDataConfig
INFO: Data import pre-processing completed in 2.89 seconds for table TI_CGSEO
_3_#lang_tag#.
Sep 20, 2013 12:40:20 AM com.ibm.commerce.foundation.dataimport.preprocess.Da
mportPreProcessorMain logExitCode
INFO:
The program exiting with exit code: 0.
Data import pre-processing completed successfully with no errors.

Sep 20, 2013 12:40:20 AM com.ibm.commerce.foundation.dataimport.preprocess.Da
mportPreProcessorMain logEndDateAndTime
INFO: Data import pre-processing ended:Fri Sep 20 00:40:20 IST 2013
Sep 20, 2013 12:40:20 AM com.ibm.commerce.foundation.dataimport.preprocess.Da
mportPreProcessorMain logEndDateAndTime
INFO: Data import pre-processing completed in 56.491 seconds.


Verify that the Program executed successfully.

Also, inspect the following file for errors:
To get more logging information, update the logging level from INFO to FINEST in the following file: 
# Default global logging level, INFO
.level=FINEST
You can also increase the log file size and the number of log files. For example:
# Limiting size of output file in bytes:
java.util.logging.FileHandler.limit=50000000

# Number of output files to cycle through
java.util.logging.FileHandler.count=2

Well now for the last step .

After preprocessing the search index data, you must build the search index.

Start the server.

 navigate to bin folder.
run the following.

di-buildindex.bat -instance instance_name -masterCatalogId masterCatalogId [-indextype indextype] -dbuser dbuser -dbuserpwd dbuserpwd [-fullbuild true | false] [-statusInterval statusInterval] [-localename localename] [-indexSubType indexSubType] [-basePath basePath] [-storeId storeId] [-webcontentDelete true | false] [-solrConnTimeout solrConnTimeout] [-soTimeout soTimeout] [-workspace workspaceId] [-force true | false


eg:  di-buildindex.bat -masterCatalogId 10051 -basePath C:\WCDE_ENT70\workspace\StaticContent\WebContent\Madisons\StaticContent\en_US

In this case i have a seperate project StaticContent.

Posted by at

8 comments:

  1. Unknown8 April 2015 at 05:15

    At a time when affordable homes seller are in short supply, the houses in Cobridge — or ‘Poundland’, as locals have dubbed it — represent a barely believable bargain.

    ReplyDelete
  2. Unknown8 April 2015 at 05:20

    This comment has been removed by the author.

    ReplyDelete
  3. Unknown13 April 2015 at 10:41

    Get Personalized Elmo CD for your children

    ReplyDelete
  4. Unknown4 May 2015 at 19:54



    CHRONIC PAIN
    Not all pain that persists will turn into chronic pain.
    Different people experience chronic pain very differently.
    Likewise, the effectiveness of a particular treatment for
    chronic pain will often differ from person to person.
    For example, a particular medication or injection
    for a herniated disc may provide effective pain relief
    for some people but not for others.

    One problem is that not all patients with similar conditions
    develop chronic pain, and it is not understood why some people
    will develop
    CHRONIC PAIN. Also, a condition that appears
    relatively minor can lead to severe chronic pain,
    and a serious condition can be barely painful at all.

    ReplyDelete
  5. onebigteam 31 May 2015 at 06:08


    Running your own virtual private server (VPS) was once limited to either profitable
    side projects or those with money to burn. The relatively high monthly costs (often $40-$60 per month)
    made it too expensive for personal projects that didn’t generate income and more serious endeavors often
    used dedicated hardware, leaving VPS as a kind of …

    ReplyDelete
  6. Unknown14 April 2016 at 00:48

    Phenq PhenQ is a all-natural dietary supplement for fat burning that combines the best qualities of many other supplements into a single pill.

    ReplyDelete
  7. Unknown2 May 2016 at 22:49

    Queens Peak MCC Land
    Pavilion & The Andrew Residences by mcc land bedok queens peak . This attractive land parcel is located within an established residential estate and next to Potong Pasir MRT station. The land parcel is located near commercial amenities such as NEX and
    Tampines Ave 10 Queens Peak . Residents can also engage in recreational activities such as water sports at Kallang Riverside. The future development is also ideal for families with school-going children, with St. Andrew’s Junior and Secondary Schools, St. Andrew’s Junior Queens Peak Bedok Reservoir .
    The 99-year leasehold site at Upper Serangoon Road/Meyappa Chettiar Road sits on about 173,830sqf of land and has a maximum allowable gross floor area of 608,408sqf. About 685 homes and 53,820sqf of commercial space can be developed on Dundee Road Queenstown Queens Peak .

    ReplyDelete
  8. welcome8 June 2016 at 20:09

    What is my ip
    An ip address is a fascinating product of modern computer technology designed to allow one computer
    (or other digital device) to communicate with another via the Internet. whatmyip addresses allow the location
    of literally billions of digital devices that are connected to the Internet to be pinpointed and differentiated
    from other devices. In the same sense that someone needs your mailing address to send you a letter,
    a remote computer needs my ip location to communicate with your computer.

    ReplyDelete

Subscribe to: Post Comments (Atom)